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ABSTRACT 


This thesis presents an application of Multidimensional Scaling 
(MDS) used in the prioritization of ASW decision functions in the 
S-3A. The ASW decision space was divided into 14 discrete decision 
functions for purposes of this analysis. The problem of developing a 
prioritization methodology was approached from two independent 
directions. 

First, an unconstrained sorting task was preformed to provide 
input to a Multidimensional Scaling algorithm. The result of this 
analysis provided a three dimensional representation of the decision 
space with dimensional interpretation. Second, a series of ranking 
tasks were preformed to provide input to an Unfolding Analysis 
algorithm. The Generalized Distance Model was selected as the model 
most representative of the ranking data. 

The decision function coordinates for the MDS algorithm and the 
decision function coefficients for the Unfolding Analysis algorithm 
were combined in a regression-like equation to provide a 
prioritization methodology for the 14 decision functions of the S-3A 


ASW decision space. 











as 


ET. 


IV. 


TABLE OF CONTENTS 


INTRODUCTION 


A. HISTORY 





Be. ASW MISSION 

C. PLATFORM OF INTERESTS 

D. PURPOSE 

E. ORGANIZATION OF THIS THESIS 
MULTIDIMENSIONAL SCALING TECHNIQUES 
A. OVERVIEW 


B. PAIRWISE COMPARISON 





C. UNCONSTRAINED SORTING 
D. TRIAD TEST 
E. COMPARISON OF TECHNIQUES 
F. SELECTION TECHNIQUE 
APPLICATION 
A. PROBLEM 
1. Unconstrained Sorting Task 
2. Ranking Task 
B. DATA COLLECTION 
DATA ANALYSIS 
A. OVERVIEW 
B. PRELIMINARY ANALYSIS 
1. Preprocessing of Sorting Data 


2. Preliminary Analysis of Rankings 





Vel 
a 
eZ 
is 
14 
14 
16 
16 
19 
20 
21 
22 
23 
24 
24 
28 
Sn 
33 
36 
36 
36 
36 


41 








C. APPLICATION OF MDS ALGORITHM 


D. DIMENSIONAL INTERPRETATION 


thes 


2 


3c 


E. PRIORITIZATION OF THE DECISION SPACE 


Ve 


Zs 


Interpretation of Dimension 
One: Workload 


Interpretation of Dimension 
Two: Time Criticality 


Interpretation of Dimension 
Three: Complexity 


Unfolding Analysis and 
Priority Mapping 


Four Models of Priority 


F., UNFOLDING ANALYSIS AND PRIORITY MAPPING 
OF DECISION FUNCTIONS 


1. Prioritization of Decision Functions 
by Importance in Mission 
with Attack 
2- Prioritization of Decision Function 
by Importance in Mission 
Without Attack 
a DISCUSSION 


A. RESULTS 


he 


2 


Summary 
Iterpretation of Results 
a. Gain Attack Criteria 


b. Classify Signal and Determine 
Signal is Valid Contact 


Comparision of Results to 
Similar Work 


47 
49 


2 


61 


62 


64 


64 
67 


71 


ou 


76 
79 
79 
79 
81 


81 


83 


86 








B. CONCLUSION 87 





C. RECOMMENDATIONS 2 
APPENDIX A 94 
APPENDIX B 102 
APPENDIX C 114 
APPENDIX D 22 
APPENDIX E 128 
BIBLIOGRAPHY L292 
INITIAL DISTRIBUTION LIST E32 








Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


Table 


i 


TL Tas 


IV. 


VI. 


VII. 


VIII. 


TX. 


XI. 


XII. 


KELL 


XIV. 


XV. 


XVI. 


LIST OF TABLES 


Decision Function Composition of Six 
Air ASW Decision Functions 


Description of the Fourteen 
Decision Functions 


Dissimiliarity Matrix 


S1qgnLiicance of 
in Ranking 


Subject Agreement 


Average Ranking of Decision in Mission 


with Attack 
Average Ranking of Decision in Mission 
Without Attack 
Average Ranking of Decision by 
Urgency 

Average Ranking of Decision by 
Workload 


Multidimensional Scaling Coordinates 


Significance of Results for Mission 
with Attack 


Comparison of Coefficients of Variation 
for Mission with Attack 


Decision Function Prioritization for 
Mission with Attack 


Significance of Result for Mission 
without Attack é 


Comparison of Coefficients of Variation 
For Mission without Attack 


Decision Function Prioritization for 
Mission without Attack 


Fourteen ASW Decision Functions and 
Alphabetic Codes 


26 


Pa 
42 


44 


45 


45 


46 


46 


25) 


72 


74 


75 


76 


ui 


78 


eS 








Figure 


Figure 


Figure 
Figure 
Figure 
Figure 
Figure 
Figure 
Figure 
Figure 
Figure 


Figure 


Figure 


Figure 


10. 
ll. 


12. 


Ss 


14. 


Lioh Or FIGURES 


Sample Decision Function 


Multidimensional Scaling Prioritization 
Procedure (adapted from Zachary 1980b) 


MDS Stress vs. Dimensionality 
Three Dimensional MDS Plot 
Dimension 1 vs Dimension 2 
Dimension 1 vs Dimension 3 
Dimension 2 vs Dimension 3 
Dimension 1 Unidimensional Plot 
DimenSion 2 UnidimensSional Plot 
Dimension 3 Unidimensional Plot 
Four Models of Priority 


Priority Score of Decision 
Functions (S~3A) 


Priority Score of Decision 
Functions (P-3C) 


Plots of Decision Functions on Three 
DimenSions (from Zachary 1980b) 


Syl 


37 
48 
51 
a 
53 
54 
56 
i 
58 
68 


82 


88 


128 








ACKNOWLEDGEMENTS 

The author wishes to express his sincere appreciation to 
the following individuals, without whose timely and generous 
assistance this work could not have been produced: Dr. Julie 
Hopson, Human Factors Branch, Naval Air Development Center; 
Dr. Wayne Zachary, Analytics, Inc., Willow Grove, 
Pennsylvania; and Lieutenant Commander Rich Hulse, VS 
Support Unit, NAS Cecil Field; all of whom contributed ideas 
and recommendations incorporated in this work. 

The author is particularly greatful to his thesis 
advisors, Commander W. E. Moroney, MSC, USN, and Professor 
D. E. Neil. Their hours of counsel, encouragement, and 
editorial assistance enabled him to form and express 
concepts herein. 

Finally, I would like to thank my wife, Carol, for her 
computer graphics assistance, data compilation programs, 
editorial assistance, love and understanding during a very 


trying period. 


10 








I. INTRODUCTION 


Pee HISTORY 

Decision making and the process by which decisions are 
made in the complex world of Antisubmarine Warfare (ASW) has 
long confounded the designer as well as the operator of 
modern sophisticated weapons systems. Operator [Zachary 
1980a], Software, and hardware [NAVAIR 1975] limitations 
have been cited as causes of less than optimal performance 
in the modern ASW mission. 

Antisubmarine Warfare has traditionally been thought of 
as a half science - half art method of locating hostile 
submerged vessels. Originally, ASW was conducted solely by 
war ships using active ranging equipment where tactical 
‘guessing’ filled the information gaps left by onboard 
sensors. As the missions of submarines became more 
diversified, air and subsurface weapons platforms were 
developed to augment existing surface weapons systems. With 
the advent of computers, the information gaps of past 
sensors narrowed considerably. However, while the operator 
received more accurate, real time information, the quantity 
of this new information could, at times, overwhelm even a 
Seasoned operator. To enhance the operator's ability to 
perform the ASW mission, systems were developed to integrate 


the large quantity of information, filter the ‘noise’ from 
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the needed inputs, and present the information in an 
orderly, useful manner. Such systems were implemented for 


surface, subsurface, and airborne weapons systems. 


B. ASW MISSION 

All ASW weapons systems - airborne, surface, and 
subsurface - share four basic phases of mission 
accomplishment: search, localization, track, and attack. The 
first phase, search, involves the planning and 
implementation of strategies (sensor placement, areas of 
Secarenw, Initial tactics) to gain initial detection of 
potential threat. In the second phase, the potential threat 
1s localized through the use of additional sensors and 
detection information. After localization of the potential 
threat, the third phase, tracking iS accomplished by the 
integration of sensor information and tactics. In peacetime, 
tracking is the mission priority, while in wartime, the 
fourth phase, attacking, receives a higher priority. These 
phases should be followed systematically to assure, in time 
of peace, constant position information on potential 
threats, or in time of war, destruction of hostile 
Submarines. 

Throughout these four phases, ASW personnel are engaged 
in decision making tasks of various degrees of complexity 
and difficulty. Decision aids could be of great use in 


performing these tasks. Antisubmarine Warfare is a dynamic 
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environment in which one can not train for all possible 
conditions. Accordingly, emphasis is placed on software, in 
the form of decision aids, to assist the operator in the 
performance of the ASW mission. However, decision aids have 
traditionally been based on designer subjectivity, random 
fleet inputs, and in response to a new, immediate threat. 
Regretably, fleet ASW personnel have had little impact on 
the design and implementation of ASW decision aids. A 
prioritization of ASW decision areas in which decision aids 
could be of benefit would directly impact on all ASW assets. 

Because of the multitude of ASW platform types in the 
U.S. Navy, the scope of this thesis was limited to one 
platform in the accomplishment of the ASW mission. The 
Criterion for selection were (1) an operator whose primary 
duty is ASW, (2) a ‘'state-of-the-art' weapons system 
(nardware and software), and (3) ease of data collection. 
The S-3A Viking aircraft fulfilled these criteria and was 


selected as the wearpons platform of interest. 


C. PLATFORM OF INTERESTS 

The S-3A Viking, a carrier based, twin engine jet 
aircraft, with an onboard digital computer, sophisticated 
avionics and crew of four iS considered to be the Carrier 
Battle Group's first line of defense from hostile 
Submarines. Although the S-3A has other missions, only the 


ASW misSion will be addressed in this thesis. The crew of 
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the S-3A consists of: Pilot, who is responsible for safety 
of flight; the Copilot, who operates the nonacoustic sensors 
(Radar, Flir, ESM, and MAD); the Acoustic Sensor Operator 
(SENSO), who is responsible for passive and active acoustic 
Surveillance; and the Tactical Coordinator (TACCO), who is 
normally the tactical mission commander and is responsible 
for the prosecution of subsurface contacts. Because the 
TACCO receives information from all other members of the 
crew as well as the onboard General Purpose Digital Computer 
(GPDC) for his tactical decision making, he will be of 
primary interest in establishing priority for decision aids. 
For definitive purposes, a decision aid is any methodology, 
algorithm, queue, or filtering mechanism that enhances an 
operator's performance by eliminating unwanted, extranious 
information and presents critical information in an orderly, 


Streamline fashion to expedite a needed response. 


D. PURPOSE 

The purpose of this work was to examine ASW decision 
making in the S-3A Viking through the use of mathematical 
modeling techniques and to establish a prioritization 
technique for the development of decision aids to assist 


decision making in the S-3A. 


E. ORGANIZATION OF THIS THESIS 


The main body of this thesis addresses the results of 
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Multidimensional Scaling techniques and the resulting 
applications to decision aids. Chapter 2 discusses’ the 
advantages and disadvantages of various scaling methods 
classified under the general heading of Multidimensional 
Scaling. The methods evaluated included pairwise comparison, 
unconstrained sorting (Q-Sort), and the triad method. 
Chapter 3 discusses the data collection. Chapter 4 contains 
the data analysis based on the unconstrained sorting 
technique with emphasis on Unfolding Analysis. Finally, 
Chapter 5 discusses the results, conclusions, and 


recommendations. 
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If. MULTIDIMENSIONAL SCALING TECHNIQUES 


A. OVERVIEW 

ASW decision making, because of the complexity and 
quantity of information flow, lends itself to few 
mathematical modeling techniques. ASW decisions, while based 
on a logical flow of events, are subjective in nature 
yielding ordinal data points at best. 

Few modeling techniques can use data of less than 
interval nature. However, a family of modeling techniques 
does exist that use nonmetric (ordinal data) as well as 
metric (interval or ratio data) inputs. One such group of 
techniques is Multidimensional Scaling (MDS). Before an 
application of Multidimensional Scaling can be used to 
examine ASW decision making, a working understanding of the 
technique should be attained. 

Multidimensional Scaling is the term loosely used to 
identify a large group of varied and powerful techniques for 
the analysis of data normally associated with the behavioral 
sciences. There exist two main purposes for utilizing these 
techniques - (1) to detect any pattern, Structure, or 
relationship that may be hidden in a matrix of empirical 
data, and (2) to represent that structure in a form that is 
intuitively appealing and is more intelligible to the human 


(i.e. a two or three dimensional graph vice a five or six 
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degree dimensionalization). The stimuli under study are 
represented by points in this geometric model such that the 
Slgnificant facets of the data points (stimuli) are revealed 
in their relationship among other stimuli [Shepard 1972]. 

The spatial representation resembles more traditional 
relationship scales such as temperature or time in that it 
attempts to acquire the fundamental properties (structure) 
of stimuli under study solely by setting them into 
correspondence with positions within a spatial subset of N 
dimensionality. It differs from simple unidimensional (i.e. 
conjoint measurement) scales in that in order to utilize all 
of the information provided by the data or capture the full 
complexity of the stimuli, the data pointS may assume 
positions within a two, or three dimensional space as well 
[Kruskal 1978]. 

In most cases, one SeekS a presentation of the model of 
the lowest possible dimensionality consistent with the 
problem to be solved and the data. Obviously, a lower 
dimensional model would be easier to understand intuitively 
and eaSler to work with, for it represents the data by means 
of a smaller number of parameters. Minimizing the number of 
parameters also yields a more reliable statistical base due 
to larger data subsets. However, the reduction of 
dimensionality is not an end unto itself. One runs the risk 
of damage to the data by arbitrarily reducing the 


dimensionality of the model. If the relationship among 
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Stimuli is sufficiently complex, two or three dimensions, 
then the model better ‘fits' the empirical data. The 
goodness of fit measurement called 'stress' is the parameter 
that measures the deviation of the empirical data from the 
model. There exist a gray area of trade-offs between easy 
visualization and the use of all data information. It is the 
analyst's responsibility to execute the trade-off after 
evaluating the criteria for analyzing the model. 

Multidimensional Scaling utilizes the same basic 
concepts as Factor Analysis. The high dimensionality, 
usually associated with Factor Analysis, is caused by the 
rigid assumption of linearity among stimuli. This assumption 
is relaxed in the model used for Multideminsional Scaling, 
allowing (normally) a two or three dimensional 
representation. Essentially, Multidimensional Scaling 
provides greater stimuli resolution in a more readily 
visualizable model than Factor Analysis. 

Multidimensional Scaling uses Similiarities or 
dissimiliarities among stimuli to define the relationships 
that exist in each possible dimension. Several subsets of 
multidimensional scales exist in various forms for different 
appilcations. The subsets include paired and triad 
comparisons, unconstrained sorting, conjoint measurement, 
delphi method, and multiple attribute utility models. 
However, the major subsets considered in this thesis were 


limited to pairwise comparison, unconstrained sorting, and 
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triad comparison because of external constraints placed on 
the data collection. The constraints were the short periods 
of time available for data collection and the 
inaccessability of subjects after the initial interview. The 
constraints will be discussed throughly in Chapter IIIf. 
Because of the multiple passes required through the data and 
subject interaction required, the techniques of multiple 
attribute utility, delphi and conjoint measurement were 
considered less desirable than the selected techniques. 

The measure of effectiveness for the pairwise 
comparison, unconstrained sort, and triad comparison is how 
well the model or technique represents the empirical data. 
This measure of effectiveness is called ‘stress‘ [Kruskal 
1978). Stress is the goodness of fit measurement for the 
model used to determine the number of dimensions considered, 
balanced by the amount of deviation from the data. Stress 
measures the degree of departure from the assumption that 
there is a monotonic relationship between the non-metric 
dissimiliarity measures of the data and the metric distance 


measures of the representational structure [Burton 1968]. 


B. PAIRWISE COMPARISON 

The most common technique is the pairwise comparison of 
stimuli with a ranking or weighting between the two [Kruskal 
1978]. The pairwise comparison examines N different stimuli 


in pairs, assigning relative ranks as to the degree of 


Ms, 





Similiarity or dissimiliarity of the two stimuli being 
compared. The comparisons are used to form matrices of 
ordinal preference data. The pair-wise data is examined for 
hidden relationships among three or more stimuli that would 
not normally come to light in pair-wise preference mapping. 
These relationships form the dimensionality of the model and 
point to areas of greatest (or least) interest. 

While the pairwise technique utilizes the greatest 
amount of information from a given set of data, several 
limitations become apparent in the use of this comparison 
method. For N stimuli, the number of pair-wise comparisons 
are N(N - 1)/2. For all but a small number of comparisons 
(N < 10), the number becomes unmanageable. The complexity 
increases with larger subject samples. Additionally, the 
tester normally has a finite time limit and even a small 
Sample (N = 8, subjects = 10) requires a large amount of 


time. 


C. UNCONSTRAINED SORTING 

The unconstrained sorting task {Burton 1975] requires 
Subjects to sort stimuli into groups. The sorting into 
groups represents a partitioning of the stimuli set. 
Normally, the stimuli set consist of the stimuli represented 
in a medium that is easily sorted (such as 3 X 5 cards) and 
test subjects are asked to sort the stimuli into groups. The 


instructions for the formation of these groups require the 


20 





subjects to sort the stimuli that are ‘similar in meaning' 
or ‘belong together' into the same group. The subjective 
interpretation of the sorting instructions by each member is 
the basis for an unconstrained sort. The subsets (groups) of 
stimuli can have any number of members. A single stimulus in 
a group represents no similarity to any other stimulus and 
all stimuli in a group represent total similarity among 
Stimuli. Test subjects who have few subgroups of stimuli are 
called ‘lumpers’ and subjects who make many distinctions 
among stimuli are referred to as ‘slitters'. Burton's study 
of Q-Sort data [1975] suggest that among Multidimensional 
Scaling techniques, the unconstrained sorting task is the 


most flexible in terms On testing procedures. 


DeeelRLAD TEST 

The triad test is similar in procedure to the paired 
comparisons discussed earlier except stimuli are compared 
three vice two at a time. The triads test may be 
administered in two different ways. The first way requires 
the subject to choose the most different stimulus from among 
the three stimuli presented. The second version of the test 
asks the subject to pick the most different stimulus (as 
before) and also to pick the pair of stimuli which are most 
different (one from another) [Burton 1975]. 

There are ( N ) or N(N = 1)(N —- 2)/6 triads for N 


3 
Stimuli. These triads are used to form matrices of ordinal 
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Mmeeceerence data. This ordinal data is monotonically 
transformed to a metric form that can be examined for 
relationship or structures that are not apparent at first 


inspection. 


E.- COMPARISON OF TECHNIQUES 

Although the pairwise comparison was the most common 
test found in the literature review (Shepard 1972], [Kruskal 
1978], the unconstrained sort and triads test revealed 
several interesting advantages for the implementation of 
either technique. The advantages and disadvantages of each 
technique are discussed in this section. 

All three techniques share the advantages of being a 
Simple task to understand and all can be presented either 
verbally or in written form. At this point, the paired 
comparision and triads test begin to differ from the 
constrained sort. For N stimuli and M subjects, the number 


of pairs presented by the tester is ( N )M or 


2 
N(N - 1)M/2 and the number of triads presented equal (2M 
or N(N — 1)(N — 2)M/6. Intuitively, one can see the 


advantage of the triads test in the sheer number of stimuli 
groups. However, for N > 10 and M > 10, either test soon 
becomes unmanageable in terms of time necessary to give the 
test. 

On the other hand, the unconstrained sorting technique 


has one clear advantage : time. The unconstrained sort can 


22 





be given to several subjects at once and recent research by 
Burton [1975] suggests that most people can do a single 
Sorting of sixty(60) stimuli in fifteen to thirty minutes. 
Additionally, experiments by Miller [1974] indicate that 
triads test and sorting task uncover the same structure 


within sorting decisions. 


EF. SELECTION OF TECHNIQUE 

As discussed -in the preceeding sections, each of the 
three Multidimensional Scaling techniques can be applied to 
a great number of problem areas with varying degrees of 
Success. An examination of the problem area suggested that 
one technique was superior to the other two. 

The problem concerned the ASW operational decision 
making environment, specifically the S-3A aircraft. Because 
of the dynamics of the S-3A environment ashore (Weapons 
systems Trainers, ground schools, and training flights), it 
was essential to interview as many experienced TACCOS as 
possible in the shortest amount of time so as not to 
interfere with training and operational commitments. 
Consequently, with time as the major constraint, the 


unconstrained sorting technique was selected. 
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III. APPLICATION 


A. PROBLEM 

The development of air ASW decision aids requires an 
understanding of the complexities and the interrelationships 
among decisions of the ASW mission. The measure of 
importance with respect to ASW decision making 1S a 
non-quantifiable variable that directly impacts. on 
priorities among decisions. Because of the subjective nature 
of ASW decision making, Multidimensional Scaling, in 
general, and the Unconstrained Sorting Task, specifically, 
lend themselves favorably to the prioritization of the ASW 
decision space. An examination of the specific S-3A ASW 
decision making process was helpful as an overview to the 
problem. 

As mentioned in the introduction, the air ASW mission 
was divided into four general phases: (1) search, (2) 
localization, (3) track, and (4) attack. While all of these 
areaS are important, the number of stimuli (phases) 
concerning the mission was too small to yield any 
appreciable information about the decision function. An 
improvement over the four phase model was the repartitioning 
by Zachary [1980a] of the air ASW decision space into six 
Specific decision situations. The six specific decision 


Situations were: 
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1) On-station search 

2) Contact classification/verification 
3) Localization 

4) Surveillance tracking 

5) Attack planning 

6) Lost contact requisition 

The six decision situations model, while an improvement 
over the four phase model, did not remedy the problem of 
limited information used as input to Multidimensional 
Scaling models. The six 'phase' model was further evaluated 
by Zachary [1980b] and found to have varying numbers of 
constituent decision functions associated with each phase. 
These constituent decision functions were not considered to 
be confined to only one decision situation but rather each 
decision function could be used in any or all combinations 
of the decision situation (phases) to which that decision 
function was considered applicable. Table I shows the 
decision function composition of the six air ASW decision 
Situations. 

From a careful examination of the decision situations in 
Table I, fourteen (14) distinct decision functions emerged 
as necessary to completely describe the air ASW decision 
space within the context of this thesis. Consequently, the 
fourteen decision functions listed in Table II became the 
stimulus set for the multidimensional scaling sampling. The 
decision functions and their descriptions are listed in 
Table II. 


25 





Table I. Decision Function Composition of Six 
Alr ASW Decision Functions 


DECISION SITUATION CONSTITUENT DECISION FUNCTIONS 


ON-STATION SEARCH CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

EXTEND SENSOR PATTERN 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 
COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


CONTACT CLASSIFICATION/VERIFICATION CLASSIFY SIGNAL 
DETERMINE SIGNAL IS A VALID CONTACT 


LOCALIZATION MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

GAIN ATTACK CRITERIA 

DETERMINE TARGET FIX 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 
COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


CREATE SENSOR PATTERN 
MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

DETERMINE TARGET FIX 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 
COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


CREATE SENSOR PATTERN 
MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

GAIN ATTACK CRITERIA 

DETERMINE TARGET FIX 

ADJUST PATTERN TO SENSOR FAILURE 

DETERMINE WEAPON AND SETTING FOR ATTACK 
CONSTRUCT SENSOR MONITORING PATTERN 
DETERMINE AIRCRAFT WEAPON LAUNCH POSITION 
COMPENSATE FOR !N-SITU PROPAGATION CONDITIONS 


CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 
COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 
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Table II. Description of the Fourteen Decision Functions 


- ADJUST PATTERN TO SENSOR’ FAILURE (AP) = 
replacement of faulty sensors or adjustment of 
alrcraft track due to requipment/sensor failure. 


- EXTEND EXISTING SENSOR PATTERN (EP) - determine 
Orientation and settings of sensors to be added to 
existing pattern with no contact. 


- ANTICIPATE TARGET MOVEMENT (AM) - predict future 
location, course, speed, and depth of target based 
on intelligence and actual tactical situation. 


- CONSTRUCT SENSOR MONITORING PATTERN (MP) - 
determine where to position the aircraft to obtain 
maximum reception from sensors, and sequence in 
which to monitor deployed (acoustic) sensors if 
number of sensors exceeds number of aircraft 
receivers. 


~ COORDINATE HAND-OFF (CH) - transmisSion of data 
to relief platform for continuation of prosecution. 


- DETERMINE SIGNAL IS A VALID CONTACT (VC) - reduce 
false alarm rates of sensor system (e.g., clouds on 
radar, random noise on MAD or acoustic). 


- DETERMINE WEAPON AND SETTING FOR ATTACK (DS) - 
Select weapon to be used for the attack, and its 
optimum weapon settings (e.g., search depth, 
minimum/maximum search limit). 


~ GAIN ATTACK CRITERIA (AC) - interpret sensor data 
to determine when target is localized sufficently 
to place an attack. 


- DETERMINE TARGET FIX (TF)- uSe incoming sensor 
data to establish the location of the target. 


~ CREATE SENSOR PATTERN (CP) -~ determine pattern, 
spacing, orientation and/or utilization for all 
sensor types for each phase of the mission. 


- MANAGE EQUIPMENT/STORES TO ACCOMODATE PRESENT AND 
FUTURE NEEDS (ME) - monitor inventory of sensors 
and equipment status for application for future 
tactics and resources. 
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Table II. Continued 
- CLASSIFY SIGNAL (CS) - determine if signal 
(primarily acoustic) may be originating from the 
target of interest. 
- COMPENSATE FOR ACOUSTIC/ATMOSPHERIC PROPAGATION 
CONDITIONS (PC) - determine the adjustments in 
sensor spacing, aircraft track, aircraft altitude, 
eC a. that must be made when actual (in situation) 
atmospheric and bathythermal conditions are 
different from their forecast. 
- DETERMINE AIRCRAFT WEAPON LAUNCH POSITION (LP) - 
determine factors to place the aircraft in an 
optimum attack position. 

In this analysis, the TACCOs were asked to perform two 
psychometric tasks to elicit indicators as to the subjective 
nature of ASW decision making. The first task was an 
unconstrained sorting of the stimulus set of 14 decision 
functions that provided the necesSary inputs for the 
Multidimensional Scaling. The second task was a set of rank 
orderings of the stimulus set that provided inputs to the 


unfolding analysis and the correlations testing. Both task 


will be discussed in detail in the following sections. 


1. Unconstrained Sorting Task 


The technique of the unconstrained sorting, also 
know as the 'Q-sort', was first developed by Stephenson 
[1953]. The technique was further refined by Burton [1968] 
and applications of the scaling were validated by Miller 
[1974]. In this task, the subject is asked to partition the 
members of the stimulus set into an arbitrary number of 


groups of unspecified size. The criterion for sorting 


28 





usually is a similarity or dissimiliarity judgement by the 
Subject established by the instructions for the task. There 
is no limit on the number of stimuli in a group nor is there 
a limit on the number of groups. 

As discussed in the previous chapter, the 
unconstrained sorting task is only one of many psychometric 
methods used to collect data for Multidimensional Scaling. 
Unconstrained sorting was chosen over the other discussed 
techniques for two reasons. First, the unconstrained sort 
allows the levels of discrimination among the stimuli by 
Sagi suoject to be controlled for explicitly in the 
processing of the results. Subjects will place varying 
degrees of emphasis on the underlying structure of the 
interrelationships among stimuli. The algorithms developed 
for this technique in the processing of data inputs control 
for between subject variance. Subjects, however, vary widely 
in the level of detail perceived to be necessary for 
performance of the sorting task. The perceived level of 
detail task performance can differentiate broad distinctions 
that find all stimuli very much alike or very different; or 
from very few distinctions that find small groups of stimuli 
Similiar but in different degrees. Depending on the degree 
of distinction required by the interviewer, the instructions 
must be expressed so as to focus on a specific level of 
distinction required of a scaling task. This is the primary 


reason for choosing the unconstrained sorting task. 
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secondly, the unconstrained sort is the simplest, 
most expeditious method of collecting data fOr 
Multidimensional Scaling. As discussed in the previous 
chapter, the limited amount of time available for TACCO 
interviews was critical. The sorting task portion of the 
interview took less than 30 minutes for each TACCO, favoring 
this technique over more time consuming methods. 

Thorough investigation of Burton [1968], Miller 
[1974] and Zachary [1980b] have shown that unconstrained 
sorting produces equivalent results to the other MDS 
techniques when two general rules of thumb are obeyed. The 
first rule is that the number of stimuli must be greater 
than ten, (S > 10), and second, there must be at least twice 
aS many subjects (N) performing the sort as there are 
Stimuli, (N vee > ee Lee EOUGEC en GeCIS TON EUnet! Os, 
previously discussed, meet the stimuli number requirement 
and a total of 30 S-3A TACCOs interviewed satisfied the 
second condition. 

Each TACCO interviewed was presented the sorting 
task stimuli verbally, in the briefing, and in writing, in 
the form of a deck of 3X5 inch index cards. The written and 
the verbal presentations were identical in nature. In the 
center of each stimulus card (in bold print) was the name of 
the decision. Following the name was a brief explanation of 
the decision function. To identify each stimulus card, a two 


letter code was used (in the upper right corner) to record 
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the sorting response 


as well as the ranking (to be discussed 


later). Figure 1 represents a sample decision card used in 


the interviews. The codes were mnemonic in nature, derived 


from the keywords of 
the codes and the 
collection. Mnemonic 
or alphabetic codes 
cards were sorted 


ordering. 


the decrision functions. Table II lists 
decision functions used in the data 
codes were selected instead of numeric 
to eliminate any bias in the way the 


due to any unintentional implicit 


Sample Decision Function Card 


a ae ae eet ee a ee? ee? ee eee ee ee ee ee ee ee ee ee ee ee es ee ee ee ee ee ee ee ee oe 


ANTICIPATE TARGET MOVEMENT 


predict future location, course, Speed, and 
depth of target based on intelligence and 
actual tactical situation 


> <> cee ee ee ee ee ee ee a eee ee ee 


2. Ranking Task 


en eee ee ee ee es es es ee es es oe es es es es a es oo oe os ee ee es 


The second psychometric task performed by the TACCOs 


was a rank ordering task. The subjects used the stimulus 


cards, discussed previously, and ranked the stimuli 
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according to four different criteria. The criteria for the 
four rankings were: 
1) Importance of the decision in the ASW mission 
with an attack on a submarine. 
2) Importance of the decision in the ASW mission 
with only surveillance of a submarine. 
3) Urgency of the decision, and 
4) TACCO workload during the decision. 

Rankings 1 and 2 provided inputs for the Unfolding 
Analysis (discussed in Chapter IV). They were selected for 
the Unfolding Analysis because they use the same criterion - 
importance of the decision to the accomplishment of mission 
- but under different circumstances - attack versus 
Surveillance. This difference was used as a reference. The 
Criterion of importance was defined as the degree to which a 
less than optimal decision would adversely impact on the 
achievement of the mission objective. 

Urgency, the third ranking criterion, was defined as 
the relative speed with which the decision had to be made, 
at a decision point. The fourth ranking criterion, workload, 
was defined only in the most general terms. Each TACCO was 
instructed to consider mental as well as physical workload. 
These last two rankings were used to provide supplemental 
information which could aid in the interpretation of the MDS 


Solution. 
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B. DATA COLLECTION 

This thesis is based on the assumption that a 
priorization of the S-3A ASW decision space must include the 
knowledge, opinion, and intuition of experienced S-3A ASW 
decision makers (TACCOsS). Multidimensional Scaling and 
Unfolding Analysis, discussed in the data analysis section 
of this thesis, provided a way for numerically prioritizing 
the fourteen decision functions previously discussed, on the 
basis of structured but subjective inputs from S-3A TACCOs. 

The primary source of experienced S-3A TACCO's for this 
analysis was VSWING 1, NAS Cecil Field, Jacksonville, 
Florida. For data collection purposes, an experienced TACCO 
was defined as an individual who had been designated an S-3A 
TACCO for at least one year or had made at least one 
deployment as a qualified S-3A TACCO. VSWING 1 was selected 
aS the sample TACCO population primarily because of the 
availability of experienced TACCOs during the limited time 
Frame of the data collection effort. 

The structure of the Multidimensional Scaling interview 
consisted of three general parts: 1) initial briefing and 
explanation, 2)sorting task, and 3) ranking task. At the 
conclusion of the interview an open-ended discussion of 
decision aids and decision making was held at the 
interviewee's option. The interview was designed to allow 
Several TACCO's to be interviewed simultaneously but 


conversation between TACCOS was prevented. The interviews 
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were designed to be completed in a two hour period, as to 
minimize sample bias due to interviewee fatigue and to 
minimize work time loss. The designed schedule was as 
follows: 
- 30 minutes were allowed for initial briefing and 
answering questions 
- 20 minutes were allocated for the psychometric 
Sorting task 
- 40 minutes were allocated for the psychometric 
ranking task 
- 30 minutes were used to discuss the interview and 
Suggested improvements in the area of decision 
ad S 4 
It should be noted that the TACCO's interviewed demonstrated 
an active interest in providing fleet inputs to an area many 
believed to be critical to the successful accomplishment of 
the S-3A ASW mission. 

At the start of each interview, an initial briefing and 
explanation was given, outlining air ASW history, purpose of 
the interview and the possible impact of decision 
prone gation on the implementation of the future decision 
a€aids in the S-3A. Questions were then answered and the 
interviewees were introduced to the two tasks they were to 
perform. Portions of the briefing are found in Appendix A. 

The actual data collection effort for the sorting and 


ranking tasks were performed as follows. After the briefing, 
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each TACCO was given a pack of cards (similiar to Figure 1) 
and a set of response forms (reproduced in Appendix B). The 
cover sheet of the response form requested background 
information on each TACCO but the name was purposely left 
off. The absence of names from the response forms preserved 
anonimity, hopefully leading to less biased responses. The 
background information allowed the interview to insure the 
requirement of experience and training were met prior to 
inclusion in the test sample. Each task had an instruction 
Sheet preceeding each response form, explaining the required 
task once again. The unconstrained sort used perceived 
Similiarities among stimuli as the criterion for group 
partitioning. The four ranking task used the criterion 
discussed previously. Individual questions during the task 
were discouraged to decrease any answer interpretation bias. 
All subjects finished the sorting task in less than 15 
minutes, while the rankings, normally, required 10 minutes 
per task. 

Following the last ranking task, each TACCO. had an 
opportunity to comment, regarding ASW decision aids in 
general, and S-3A decision aids in particular. Comments were 
Solicited concerning the interview (new ideas, format, 
procedure) and are discussed in the Result and Conclusion 


sections of this thesis. 
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IV. DATA ANALYSIS 


A. OVERVIEW 

Chapter Four describes the data analysis for this 
effort. Initially, the sorting data was preprocessed using 
Burton's [{1975] algorithm into a dissimiliariity matrix to 
be used as input into the MDS program. The ranking data was 
analyzed through the use of correlation techniques and 
average rankings were found. The dissimiliarity matrix was 
then input into the MDS program and a dimensional 
representation of the decision space was produced. 
Dimensional interpretations were derived. Next, Unfolding 
Analysis was used to provide a representative model of the 
ranking decision space. Finally, a priority scale of the 
decision space was established using the model coefficients 
of the Unfolding Analysis and the Multidimensional Scaling 
coordinates. Figure 2 provides an overview of this 


methodology. 


B. PRELIMINARY ANALYSIS 


1. Preprocessing of Sorting Data 


Sorting data were obtained from 32 experienced S-3A 
TACCOs, however, two TACCOs were excluded because they did 
not have the experience level required for the analysis. The 


remaining 30 TACCOs were determined to be ‘experienced’ for 
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the purposes of this analysis. These 30 sortings are listed 
in Appendix C. The 30 sets of data were prepared for input 
into the MDS computer program by METRIC, an algorithm 
developed by Burton (1975). The algorithm processes the 
unconstrained sorting data to produce a dissimiliarity 
meaSure among the decision functions in the stimulus Set 
[Zachary 1980b]. In general, a dissimiliarity measure is a 
binary mapping from the set of objects scaled to some 
numerical scale. For any two objects i and j, the 
dissimiliarity between i and j is expressed as d;.. 


13 
Dissimiliarity measures has three main properties: 


1) qi; = 0 iff: i=j : positivity 
qi; > 0 otherwise 
2) dis = di for all i, j : Symmetry 
3) qi; < dj; a day fOuscal in jy. Kees 0 Erlangle 
inequality. 


Dissimiliarity measures are normally represented as 
a square matrix where each row/column represents one of the 
Stimuli. Consequently, the terms dissimiliarity 'measure' 
and dissimiliarity ‘matrix' are use throughout the analysis 
interchangeably. 
Burton's algorithm computes disSimiliarity between 
two stimuli based on two inputs: 
1) the number of subjects who place them in 


different groups, and 
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2) the values assigned to the parameters a 
and € . 

The first parameter, a , represents the level of 
discrimination selected for this analysis (as discussed in 
Chapter III). The parameter, ¢€ , represents the minimal 
dissimiliarity between two stimuli that results when two 
Stimuli are placed in the same group by one subject. The 
algorithm is based on the principle that each subject infers 
a partition on the set of stimuli. R; is defined as the 
number of groups formed by subject i and M, is defined as 
the number of stimuli placed in group k by subject i. If the 
number of subjects is T, then dissimiliarity between stimuli 
x and y is: 

die = > (C - S; (x, y)) 
where: 
| A., if subject i places stimuli x and 
y in cell k of the partition, 
S: (x, y) = B; if subject i places stimuli x and y 


in different cells of the partition, 


d(x, y) is a measure of dissimiliarity which, in addition, 
is a metric [Burton 1975]. There are two constraints on a: 


1) B; < min (A;,) over all k 
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Zac > max (A;,) Over. alilmr, 


Additionally, a useful measure is obtained by defining: 


Ain = (My) 
Bix = 0 
c = 1]1+e¢e« for ae=0Q0 


N+ € for a> Q 


2+¢€ for a€¢ 0 


Since epsilon ( ¢€ ) is a minimum value used to 
differentiate items placed together by a given subject, it 
Should be very small. Alpha (a) represents the level of 
distinction used by the TACCO doing the sorting. If alpha is 
greater than zero (@> 0), a greater weight is given to 
Sortings that use a few, large cells, and the distance 
measure tends to be the lower dimensionality when subjected 
to Multidimensional Scaling. When alpha is less than zero 
(@< 0), a greater weight is given to partitions which 
include a greater number of small cells, and the measure 
results in higher dimensional MDS solutions. When alpha 
equals zero (a@= 0), equal weight is given large and small 
cell partitions, as 1 is always added to d(x,y) whenever x 
and y are in different cells, and epsilon is added to d(x,y) 
when they are place in the same cell. 

The program, METRIC, performs this algorithm to 


Produce a dissimiliarity matrix from the unconstrained 
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Sorting. Because the intent of this thesis is to identify 
the most important dimensions vice all dimensions, an alpha 
value of -.5 was used. The dissimiliarity matrix is shown 


in Table III. 


23 Preliminary Analysis of Rankings 


The rankings of the 14 decision functions by the 
four criteria discussed in Chapter 3 are given in Appendix D 
for all 30 TACCOs. The requirement of the preprocessing of 
data, necessary in the sorting data, was not considered 
necessary for input into the Unfolding Analysis procedure. 
However, two preliminary analyses of the ranking data were 
considered useful to the overall analysis. 

The first preliminary analysis tested the 
Statistical significance of the ranking data. The Kendall 
coefficient of concordance : W is a statistic which measures 
the relation (degree of agreement) among several rankings 
of the same set of stimuli [Seigal 1956]. The coefficient of 
concordance iS an index of the divergence of the actual 
agreement shown in the data from the maximum possible 
(perfect) agreement. A test of significance of the 
coefficient can be based on the value of W [Kendall, 1962]. 
This test is based on the fact that K(N - 1)W is distributed 
according to the Chi-Square distribution with N - 1 degrees 
of freedom (where K is the number of rankers and N is the 
numoer of stimuli). The null hypothesis, Hos and the 


alternate hypothesis Hy used were: 
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Hp : there is no agreement among the subject's 
rankings (i.e. W = 0). 

H, : there is significant agreement among the 
subject's rankings (i.e. W> 0). 

Table IV lists the coefficient of concordance : WwW, 
ene x? Statistic, and the level of significance for each of 
the four rankings. All four rankings were found to be 
Significant at less than the .01 level, rejecting the null 
hypothesis with greater than 99 percent confidence. The 
Significance testing indicates that: 

1) the TACCOs used the same criterion for ranking, 
and 
2) the response were not of a random nature. 

The second preliminary analysis of the rankings 
provided insight into the interpretation of each criterion 
(i.e. l)importance to mission with attack, 2)importance to 
mission without attack, 3)urgency, and 4)workload). Average 
‘rankings were used as a tool for this analysis. Siegal 
[1956] and Kendall [1962] suggest that the best estimate of 
the ‘true' ranking of N objects is provided, when W is 
Significant, by the order of the various sums of the ranks. 
An extention of this estimate is the average ranking. 
Average rankings yield more information about relative 
ranking among stimuli than does the sum of ranks [Zachary 
1980b]. The average rankings are shown in Tables V through 


VIII. The tables show a significantly different ranking for 
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each criterion, verifying the assumption that more than one 
Criterion was necesSary to capture the Mod elt 


prioritization of the decision functions by the TACCOs. 


Table IV. Significance of Subject Agreement in Ranking 


RANKING KENDALL CHI-SQUARE SIGNIFICANCE LEVEL 
CRITERION COErrrcrieNnr STATISTIC (FOR REJECTION OF H) 
IMPORTANCE 

WITH aoi2 148.79 < .001 

ATTACK 
IMPORTANCE 

WITHOUT ~468 182.56 S010" 

ATTACK 
URGENCY 2958 Ze OZ < .O00l1 
WORKLOAD 243 94.80 <a OL 
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Table V. Average Ranking of Decision in Mission 
With Attack 





Gain Attack Criteria 3.866 
Classify Signal 4.700 
Determine Tarrget Fix 5.400 
Determine Aircraft Weapon Launch 

Position 5.450 
Determine Weapons Setting For 

Attack 5.800 
Anticipate Target Movement 6.200 
Determine Signal Is Valid Contact 6.566 
Create Sensor Pattern 6.600 
Compensate For Propagation Conditions 8.500 
Construct Sensor Monitoring Pattern 8.666 
Extend Sensor Pattern LO a033 
Manage Equipment and Stores 10.200 
Adjust Pattern to Sensor Failure 10.566 
Coordinate Hand-Off 2 555 





Table VI. Average Ranking of Decision in Mission 
Without Attack 





Classify Signal 3.900 
Anticipate Target Movement 4.966 
Create Sensor Pattern 5.066 
Determine Signal Is Valid Contact S66 
Determine Target Fix 5.266 
Coordinate Hand-Off 5.666 
Compensate For Propagation Conditions 6.500 
Construct Sensor Monitoring Pattern 1423p 
Manage Equipment and Stores 7.766 
Extend Sensor Pattern 8.366 
Adjust Pattern to Sensor Failure 827 353 
Gain Atack Criteria 10.600 
Determine Aircraft Weapon Launch 

Position a0 S 
Determine Weapon and Setting 

For Attack 13.066 








Table VII. Average Ranking of Decision by Urgency 





Determine Aircraft Weapon Launch 


Position 2.666 
Gain Attack Criteria 2-933 
Determine Weapon and Setting 

For Attack 4.800 
Anticipate Target Movement 5.266 
Determine Target Fix S033 
Determine Signal Is Valid Contact 5.766 
Classify Signal 6.866 
Adjust Pattern to Sensor Failure 8.400 
Manage Equipment and Stores 9.366 
Compensate For Propagation Conditions 9.900 
Extend Sensor Pattern 10.266 
Create Sensor Pattern 10.666 
Coordinate Hand-Off 11.366 
Construct Sensor Monitoring Pattern i933 





Table VIII. Average Ranking of Decision by Workload 





Gain Attack Criteria 4.266 
Determine Target Fix 4.733 
Anticipate Target Movement 553 
Determine Aircraft Weapon Launch 

Position 5.566 
Determine Weapon and Setting 

For Attack 6.500 
Coordinate Hand-Off 7.000 
Create Sensor Pattern 7.300 
Manage Equipment and Stores 7.600 
Extend Sensor Pattern 8.166 
Adjust Pattern to Sensor Failure 8.633 
Compensate For Propagation Conditions 9.100 
Construct Sensor Monitoring Pattern 10.166 
Determine Signal Is Valid Contact 10.333 
Classify Signal 10.333 











C. APPLICATION OF MDS ALGORITHM 

The dissimiliarity matrix discussed in the preliminary 
analysis was the primary input to a MDS program. The 
analysis was conducted on the University of Pennsylvania 
Wharton School's DEC-10 computer using the MDSX package of 
Multidimensional Scaling programs [Coxon et al 1977]. This 
package contains several different algorithms including the 
MINASSA algorithm [Lingoes and Roskam 1975], the INDSCAL 
algorithm [Carrol and Chang 1970] and the TORSCA IV 
algorithm [Young 1968]. Zachary [1980b] used each of these 
algorithms on Similar ASW data to test for convergent 
validity among the programs, resulting in virtually 
identical solutions. Therefore, in the dimensional and 
prioritization analysis, the MINASSA algorithm was used 
because the other two algorithms are quaSi-metric, whereas 
MINASSA is totally nonmetric, that is, useful in analysing 
ordinal data. 

The MINASSA algorithm was used to construct solutions 
from one to five dimensions. The plot of the goodness of fit 
measure ‘'stress' versus dimenSion number, shown in 
Figure 3, indicates that a three dimensional solution is 
best. Kruskal [1978] suggest that if the number of stimuli 
considered, N, is greater than four times the dimension 
number, D, then a stresS meaSurement of -02 or less is 


considered a very good fit. Additionally, Roskam's [1977] 
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Figure 3. MDS Stress vs. Dimensionality 
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"rule-of-thumb' for assessing the stress of a MINASSA 
solution indicates that a stress between .05 and .01 is 
considered very good quality. Therefore, with a stress of 
-018, the three dimensional solution satifies the stress 
requirement for a good fitting model. Although the 4th and 
Sth dimensional solutions also provide stress measurements 
less than .02, the incresed complexity of the solutions due 
to higher dimensionality did not merit further 
consideration. 

The three dimensional solution is shown in Figure 4. For 
ease of interpretation, two dimensional plots of each 
possible pair of dimensions are given in Figures 5 through 
7. The plots represent relative locations in two dimensions 
of each of the 14 decision functions. Table IX shows the 
actual coordinates of each decision function in three 


dimensions. 


D. DIMENSIONAL INTERPRETATION 

The MINASSA algorithm reconstructs the underlying 
features of relationships of the decision functions as the 
dimensional axes of the MDS solution. Unfortunately, the 
algorithm cannot identify the meaning of these dimensional 
axes. The meanings of the axes must be interpreted through 
careful examination of the axes individually, through 
unidimensional projection of each coordinate set, and the 


interaction of the axes. The unidimensional projections are 


49 








shown in Figures 8 through 10. 

The multidimensional representation constructed by 
MINASSA iS unique and translation or rotation of the axes 
used to obtain a more interpretable set of projections did 
not change the relationship among sitmuli. However, the 
interpretations of the dimensions, discuss subsequently, are 
based on the "unrotated = untranslated' Spatial 
representations given in Figures 5 through 7. The basis for 
this representation is two-fold. First, the MINASSA solution 
is based on an implicit rotation performed by the algorithm 
to minimize dependence among dimensions. The orientation of 
the dimension axes provided the most independent 
representation in terms of the set of decision functions. 
Consequently, the independent nature of the resulting axes 
enhanced the interpretability of the dimensions. Secondly, a 
rotation and translation of the axis was attempted to 
provide a more interpretable representation. This analysis 
did not add to the interpretability of the spatial 
representation. Additionally, because the unrotated 
orientations is based on the criterion of minimizing the 
interdimensional correlation, the original MINASSA result is 


used in the following interpretation. 
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Table IX. Final Multidimensional Scaling Coordinates 


a 


Dimension Dimension Dimension 
1 3 

Adjust Pattern to 
sensor Failure -0447 =.5/91 22632 
Extend Sensor Pattern Ely 2 sO o20 -.5211 
Anticpate Target 
Movement -.2909 - 7820 -.2064 
Construct Sensor 
Monitoring Pattern 7 20 -.6017 -.2467 
Coordinate Hand-Off 1.4256 -.4035 ~6366 
Determine Signal Is 
Walid Contact -1.3964 ~0029 o 2927 
Determine Weapon 
Setting For Atack -6826 -6964 ~0826 
Gain Attack Criteria ©4285 ~7912 -.0787 
Determine Target Fix =. 5060 - 7070 -.1407 
Create Sensor pattern ~2664 -~.5176 =.90 28 
Manage Equipment 
and Stores -0169 -.4266 Oe 
Classify Signal Slee Sor) -.0282 oo 
Compensation for 
Propagation Conditions -.2605 =—0257 -. 3427 
Determine Aircraft 
Launch Position -6336 ~ 7694 -.0585 
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1. Interpretation of Dimension One: Workload 

This dimension was interpreted as representing the 
workload of the TACCO (physical and cognitive) when 
executing the decision function (Figure 8). The extreme 
points of this mapping - Coordinate Hand-off, Classify 
Signal, and Determine Signal is Valid Contact lend 
reinforcement to this interpretation. Although the TACCO is 
ultimately responsible for the successful completion of the 
tactical mission, the sensor operators (acoustic and 
nonacoustic) are required to Classify Signal and Determine 
Signal is Valid Contact. The TACCO may verify the sensor 
operator's decision but as far as workload of those decision 
functions to the TACCO is concerned, the workload is 
minimal. The workload of these decision functions lie in the 
interpretation and analysis of target signals, which is more 
of a sensor operator's function. On the other extreme is 
perhaps the heaviest workload decision function of the 
decision set - Coordinate Hand-off. Many inputs are required 
to be processed and assimiliated to provide a well executed 
Coordinated Hand-off. Although the format of a Coordinated 
Hand-off is well established, the degree of detail required 
to maintain target contact is great. 

The implicit ordering of decision functions between 
these extreme stimuli support the workload interpretation of 
this dimension. The next three highest decision functions 


after Coordinated Hand-off - Determine Weapon and Setting, 


ao 





Determine Aircraft Weapon Launch Position, and Gain Attack 
Criteria - involve great complexity. Because the actual 
depth of the target is difficult to assess, the TACCO must 
use every input to the problem to determine the correct 
weapon Selection and its search setting in order to maximize 
probability of kill. Maneuvering the aircraft into weapons 
launch position and gaining attack criteria must occur 
within a small window in time in order to place a weapon on 
target. A small miscalculation in the execution of the 
decision functions could cause the aircraft to be out of 
position when attack criteria is gained resulting in a lost 
attack opportunity. 

A cluster of five decision functions concerning 
sensor placement, are found close to the zero point - Create 
sensor Pattern, Construct Sensor Monitoring Pattern, Extend 
Existing Sensor Pattern, Adjust Pattern to Sensor Failure, 
and Manage Equipment/Stores to Accomodate Present and Future 
Needs. All of these decision functions exhibit two levels of 
cognitive workload. The more cognitive level concerns the 
situational inputs of each function - the decisions that 
Change with time, both present and future. The less 
cognitive level represents the computer controlled decision 
inputs such as monitoring sequence, search pattern, and 
Stores remaining, and the formatted procedures established 
by the On-Scene Commander (such as buoy spacing, type of 


pattern, and buoy channel numbers). 
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The three remaining decision functions - Compensate 
for Acoustic/Atmospheric Propagation Conditions, Anticipate 
Target Movement, and Determine Target Fix require less work 
with respect to the TACCO than the ‘zero' cluster but more 
work than the extreme Signal processing functions. The 
computer on board the S-3A has a bathythermal and a radar 
range program that takes external inputs to establish 
Situational conditions. Anticipating Target Movement can be 
assisted by the prediciton and ranging computer functions 
and fixing information is supplied by the sensSor operators 


with confidence levels. 


26 Interpretation of DimenSion Two: Time Cemelcauniy 


Dimension two was interpreted as representing the 
degree of time criticality of the decision functions. The 
unidimensional representation (see Figure 9) shows three 
well defined groupings of decision functions, helpful in 
analyzing this dimension. The most neutral decision 
functions on the criticality dimension are Deterimine Signal 
is a Valid Contact, amd Classify Signal. These signal 
processing functions are considered somewhat time critical 
in determining the tactics to use against the target but 
less critical concerning the prosecution of the target. 

On the high Critica lity extreme of the 
representation, the actual tactical decision functions - 
Gain Attack Criteria, Anticipated Targert Movement, 


Determine Aircraft Weapons Launch Position, Determine Target 
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Fix, and Determine Weapon and Setting for Attack - are 
closely clustered. The relative differences between the 
decision functions were very small numerically, suggesting 
little difference in the degree of criticality. However, the 
dynamic nature of these decision functions’ requires 
expeditious scrutiny of all time critical inputs to reach 
the most optimal decision. 

The remaining decision functions were clustered at 
the low criticality extreme of the representation. This 
stimuli subset ~ Coordinate Hand-of€£, Manage 
Equipment/Stores to Accomodate Present and Future Needs, 
Create Sensor Pattern, Extend Sensor Pattern, Adjust Sensor 
Pattern to Sensor Failure, Construct Sensor Monitoring 
Pattern, and Compensate for Acoustic/Atmospheric Propagation 
Conditions - is very important in the accomplishment of the 
ASW mission. However, time criticality is not normally a 
Factor in making any of these decisions. 

speed of decision making can be considered 
synomymous with time criticality for this dimensional 
interpretation. Figure 9 illustrates areas requiring fast, 


moderate, and more deliberate decision making. 


3. Interpretation of Dimension Three: Complexity 


Dimension three was interpreted as referring to the 
degree of complexity in the decision function with respect 
to the TACCO. Figure 10 shows a rather scattered 


unidimensional plot of the decision function. Two small 
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clusters on either extreme and an evenly distributed 
grouping of the remaining decision functions about zero made 
this dimension the most difficult to interpret. 

The decision functions on the high extreme of 
Complexity = Coordinate Hand-of€, and Manage 
Equipment/Stores to Accomodate Present and Future Needs - 
showed high degrees of decision complexity, assembling 
information for hand-off and inventorying sensors. While on 
the low extreme of the complexity dimension, the stimuli - 
Extend Sensor Pattern and Create Sensor Pattern - required 
little original thought and simply followed standard tactics 
procedures. The complexity of either decision function was 
minimal. 

The remaining decision functions are evenly spaced 
around zero suggesting moderate complexity. The stimuli - 
Classify Signal, and Determine Signal is a Valid Contact, - 
has a greater degree of complexity, processing all inputs to 
those decision functions. On the other end of the grouping, 
Compensate for Acoustic/Atmospheric Propagation Conditions, 
minimizes complexity through computer-assistance programs. 
The rest of the 'zero' grouping involve implicit degrees of 


complexity. 
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—E. PRIORITIZATION OF THE DECISION SPACE 
The three dimensions constructed by the MDS analysis of 
the sorting data represent a geometric model of experienced 
S-3A TACCOs' understanding of Air ASW decisions. These three 
dimensions represent the basic implicit relationships that 
TACCOs used in the Air ASW decision making. Therefore, other 
relationships or characteristics should exist only as 
combinations of these underlying features. One such 
characteristic is priority or importance. Two of the rank 
orderings, discussed earlier, had as criterion - importance 
to the mission. This section discusses the implicit 
combination of these three dimensions used by the TACCOs 
When they ranked the 14 decision functions using the 
importance criterion (both with attack and without attack). 
The basic technique used to determine TACCO priority 
functions is termed Priority Mapping, and has been developed 
directly for this type of application by Zachary [1980b]. It 
is related to the psychometric techniques Known as 
Preference Mapping [Carrol and Chang 1967] and is based on 
Coombs [1950] and Bennett and Hayes [1960] notion of 


Unfolding Analysis. 


{ 


1. Unfolding Analysis and Priority Mapping 
In the multidimensional model of the Air ASW 


decision space, each decision function in the stimulus set 


has a unique projection or coordinate on each of the 


64 





dimensional axes. Each dimension is considered to be a scale 
which measures some relevent perceptual attribute of the 
decision space. Collectively, the three scales in this model 
measure all of the independent features needed to represent 
the perceived interrelationships among decision functions in 
the decision space. In this model, the three dimensions 
provide numerical measures of all the important features on 
which experienced S-3A TACCOs perceive ASW decisions (as 
Shown in Table IX). 

Therefore, other salient characteristics of the 
decision functions must be combinations of the features 
represented by the axes of the MDS model. These derivative 
characteristics or features may have differing measurement 
scales, resulting in different representations within the 
multidimensional space. The measurement scales include 
categorical, ordinal, and interval-ratio. The categorical 
scale partitions the multidimensional space into unordered 
regions. The ordinal scale provides a graded or ordered 
partition of the multidimensional space. Finally, the 
interval-ratio scale represents the feature as a directed 
vector or curve passing through the multidimensional space. 

Unfortunately, little can be done to represent 
categorical derivative features in the multidimensional 
model. However, through the use of Unfolding Analysis, it is 
possible to construct a representation of ordinal or 


interval-ratio derivative features. The technique is based 
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on the concept that the general form of the representation 
of a derivative feature in a predefined multidimensional 
Space can be determined by comparing different models of 
increaSing complexity [Zachary 1980b]. 

Unfolding Analysis restricts itself to four specific 
formulae which have simple mathematical forms and direct 
psychological interpretations. Although the analysis could 
be performed on either ordinal or interval-ratio scale 
features, the analysis used in this discussion was limited 
to ordinal scales, consistent with the (ordinal) ranking 
data. 

In Unfolding Analysis, the four models are 
considered and compared in terms of their ability to 
represent a given ordinal scale in a multidimenSional space. 
Usually, the ordinal scale is a ranking of a stimulus set by 
individuals according to some criterion and the 
multidimensional space is a result of some MDS analysis. 
Unfolding Analysis attempts to represent the ranking 
criteria as a function of the MDS dimensionalization. 

Techniques, similar to regression, can be used to 
define an equation which models the criterion representation 
as a function of the MDS dimensions. Carroll and Chang 
[1977] developed a set of computational procedures for 
performing these regression-like analysis. In this analysis, 
their computational approach is used to model data on 


perceived priority of decision functions. This adaptation of 
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the model is appropriately termed ‘Priority Mapping', since 
it constructs a mathematical 'map' of the TACCOs intuitive 


prioritization of the decision functions. 


2. Four Models of Priority 
The Unfolding Analysis and Priority Mapping 


procedures employ four models of priority. These models are 
based on a heirarchial structure such that each model is a 
Special case of the next ‘'higher' model. Each model in the 
heirarchy subsumes all subordinate models. The models will 
be explained in terms of ordinal representations consistent 
with the data set. 

An ordinal scale in a multidimensional space divides 
the space into graded regions, such that a point in one 
region has a different rank than a point in another region, 
but the same rank as another point in the same region. In 
other words, the space is broken into iSopriority regions 
separated by PSOULLORICY contours. A geometric 
representation of the four models showing the isopriority 
Contour structure is shown in Figure 1l. For simplicity, 
Figure 11 and the following discussion are expressed in 
terms of a two-dimensional space. A generalization of this 
approach can be extended to three or more dimensions. 

The first model (and lowest in the heirarchy) is the 
vector model. This model assumes that each dimension 
contributes Pee a linear fashion resulting in a 


Prioritization defined by stimuli projections onto a vector 


67 






X4 


. Xy : 
: A 
A 
x 
E / m IDEAL ; 2 
yA POINT 
“ ae ae : 


AL 


D 


D 


INCREASING 
PRIORITY 


VECTOR MODEL ~ aib DISTANCE MODEL 


x4 


\ 


8 
[Sy mK ; 
IDEAL 
x 


x 
IDEAL 
POINT 


( 





WEIGHTED DISTANCE MODEL GENERALIZED DISTANCE MODEL 


Piguce li. sa four Models of sprilormity 


6§ 








in the multidimensional space. The isopriority contours are 
lines (in this case) perpendicular to the stimulus vector 
(shown in Figure lla). The vector model is similiar to the 
concept of Multiattribute Utility (MAU). Each dimension 
(attribute) contributes to the model in a linear way with 
the relative contribution of each dimension defined by a 
coefficient assigned to it. The vector model can be compared 
to the economic's ‘more is better' model, because it assumes 
that whatever the dimension, more of it will contribute to a 
higher priority. 

The remaining three models define priority in terms 
of distance in the multidimensional space. These models are 
based on the assumption that there is an ideal point 
somewhere in the modeled space, such that the closer a 
Stimulus is to the ideal point, the higher the priority of 
that stimulus. The simplest of the distance models is termed 
the Unweighted Distance Model or just Distance model. This 
model prioritizes stimuli strictly according to their 
distances from the ideal point. This distance measurement 
for this model is Euclidean in nature yeilding isopriority 
Contours represented by concentric circles around the ideal 
point (as shown in Figure 1lb). The unweighted model assumes 
each dimension has an equal contribution to priority. 

A more general representation in terms of distance 
allows the distance between the ideal point and each 


decision to be a function of weighted dimensionality. More 
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specifically, dimensions may not contribute equally to the 
model so each dimensional axes is given a relative weight. 
This model, termed the Weighted Distance Model, allows the 
isopriority contours to assume the general slope of an 
ellipse parrallel to the dimensional axes as shown in Figure 
iC . 

In the models previously discussed, an assumption is 
that each dimension contributes independently EO prrotlty. 
However, interaction may occur among some or all dimensions 
in constituting the concept of priority. To allow such 
interactions among dimensions, a model can be constructed by 
permitting the axes to be rotated before the distances are 
computed. This model, termed the Generalized Distance Model, 
allows each dimension to contribute independently and 
Collectively to the priority of an decision function. The 
isopriority contours are similar to the weighted distance 
model except the contours can asSSume any orientation in the 
Space as shown in Figure lld. 

These four models - the vector model, the distance 
model, the weighted model, and the generalized model - are 
the mathematical basis for the Unfolding Analysis and 
Priority Mapping. In Unfolding Analysis, the ability of each 
model to replicate the ordering of the decisions by the 
TACCOs, iS evaluated and the most applicable model is 
selected. In Priority Mapping, the specific coefficients of 


the selected model are determined and a precise mathematical 
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prepresentation of priority is constructed. 


F. .UNFOLDING ANALYSIS AND PRIORITY MAPPING OF DECISION 
FUNCTIONS 


Unfolding Analysis procedures were conducted for the two 
sets of ranking of decision functions by importance. The 
first analysis considered the rankings by importance in the 
ASW mission where the goal was an attack on the submarine. 
The individual rankings are in Appendix D, and the average 
rankings are given in Table V. The second analysis 
considered the rankings by importance to the ASW mission 
where the goal was surveillance of the submarine. The 
average rankings for this criterion is given in Table VI and 
the individual rankings are in Appendix D. The priority 
mapping was conducted with the PREFMAP program developed by 
Carrol and Chang [1967, 1977], which performs the Unfolding 
Analysis as described in the last section. PREFMAP, like 
MINASSA, is a part of the MDSX package of Multidimensional 
Scaling programs [Coxon et al 1977] available on the Wharton 
School's DEC-10 Computer. PREFMAP performs Unfolding 
Analysis by conducting seperate analysis of each individual 
ranking against the multidimensional configuration of the 


Stimulus set provided by the user. 


1. Prioritization of Decision Function by Importance in 
MisSion with Attack 
Using the PREFMAP program, the analysis began by 


constructing trial representations of the rankings criterion 
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using each of the four models discussed earlier. These 
representations produced different rankings of the decisions 
functions, which can be compared to the actual TACCO 
rankings to determine the appropriateness of each model. The 
Simplest, most reasonable measure of comparison is the 
correlation between the actual rankings and the rankings 
produced by the model. The correlation, when squared, 
becomes the coefficient of variation, 2 indicating the 
proportion of variation in the TACCO rankings accounted for 
in the PREFMAP rankings. A F test statistic can be computed 
from a and used to test the null hypothesis, Ho! that the 
value of r? is strictly a random result. The values of r’, 
the F ratio, and the levels of significance are given in 


Table X. 


Table X. Significance of Results for Mission with Attack 


MODEL r° EF STATISTIC REJECTION LEVEL 
GENERALIZED ~996 He Oicne <.O0l 
WEIGHT DIST. 982 62.6 eid: 
UNWEIGHTED 943 37.4 <.Ol 
DISTANCE 
VECTOR seo Ss Z <.O1 


All four models produced a significant (non-chance) 
representation of the TACCO data. The next phase of the 
analysis is to determine which is the best representation. 

The most widely used criterion for selecting among 
models is the goodness of fit measurement between the model 


and the data. The goodness of fit is measured in all four 
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models by r? ~ the coefficient of variation. The r2 value 


for all four models is greater than .99, indicating a good 
fit by all models. In fact, each model in the heirarchy 
accounts for more variation than the lower models. This 
reinforcing result is an artifact of the PREFMAP algorithm 
Since each heirarchial model increases the complexity of the 
equation with increases in the number of parameters. 
Consequently, the model with more parameters will explain 
more of the variance than the subordinate models. Therefore, 


4 will increase as the 


it is reasonable to assume that r 
heirarchy is traversed from the vector to the generalized 
model. The fundamental question is whether or not’ the 
increase in r?, as the model heirarchy is traversed, is 
Significant as the number of model parameters increase. 

To answer this question, a statistical comparison 
among the model rankings was used. A F statistic was 
computed from the r? values of each pair of models (the 
between-model coefficient of variation) to test the null 
hypothesis that the increase in the value of r? as the model 
heirarchy is traversed is only a chance result. A failure of 
this hypothsis for a given pair of models indicated that 
there was not a significant difference in the explanatory 
power of the two models and the subordinate model would be 


preferred because it is less complex. The comparison among 


the four models is shown in Table XI. 
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Table XI. Comparison of Coefficients of Variation for 
Mission with Attack 


MObPE °06—0UC—“—t=tsti‘<‘<i<‘<i‘<i‘<i‘i‘i‘i‘i‘i‘(‘(i(i‘(;*™*™:C SC nk 
F STAT 
SIGN. 
WEIGHTED opal 
.022 
UNWEIGHTED 11.2 7.4 
018 .019 
VECTOR “10.1 oo 6 21.0 
021 030 424 
GENERALIZED WEIGHTED UNWEIGHTED 


As illustrated in Table XI, the generalized 
distance model is significantly more powerful than the other 
models with a confidence of greater than 95 percent. 
Therefore, the generalized distance model was selected to 
represent the TACCO rankings according to this criterion. 

Having selected the 'best' model for representing 
the ranking criterion, a precise priority function was 
derived using the PREFMAP program. The importance of 


priority of a decision function j in a mission with attack 


(ome (d.)) was given by: 
Pyg(ds) = -2148x°5) -1.74x755 -.417x75, 
+.330X51%59 4.774% 51X33 —1.20x5 5X53 
+.018xX5) ~-091x54 ~- 670X543 eee) 


where Xa is the coordinate of d. on dimension 1 
Xi lis the coordinate of d- on dimension 2 and 
X53 is the coordinate of d. on dimension 3. 


The values of Kaye X3o0 X43 are given in Table IX. 
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The priority values of the fourteen decision function 


generated by this function are given in Table XII. 


Table XII. Decision Function Prioritization for 
Mission with Attack 


PRIORITY NAME PRIORITY 
RANK SCORE 
il GAIN ATTACK CRITERIA (AC) -.2489 
2 CLASSIFY SIGNAL (CS) -.2464 
3 DETERMINE SIGNAL IS VALID 
CONTACT (VC) -.1947 
4 DETERMINE AIRCRAFT WEAPON 
LAUNCH POSITION (LP) -~.1852 
> DETERMINE WEAPON AND SETTING 
FOR ATTACK (DS) ie 
6 ANTICIPATE TARGET MOVEMENT (TM) -.1431 
7 DETERMINE TARGET FIX (TF) =. 195 
8 CREATE SENSOR PATTERN (CP) 0402 
9 CREATE SENSORING MONITORING 
PATTERN (MP) ~0444 
10 EXTEND SENSOR PATTERN (EP) s0o7es 
id COMPENSATE FOR PROPAGNATION 
CONDITIONS (PC) 70S 
12 ADJUST PATTERN TO SENSOR 
FAILURE (AP) ~ 1867 
a3 MANAGE EQUIPMENT AND STORES (ME) ~ 1980 
14 COORDINATE HAND-OFF (CH) ~ 3656 


These values represent the relative priority of the 
decision functions in a misSion with attack. The lower 
values (i.e., negative) in Table XII indicate a higher 
priority and the higher values represent a lower priority. 
This ordering is the result of the TACCOs assigning lower 
ranks to the more important decisions and higher ranks to 
the less important decisions (PREFMAP program merely 


maintains this directionality in its computations). 
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2. Prioritization of Decision Function by Importance in 
Mission Without Attack 


An identical analysis was conducted for the TACCO 
rankings of decision functions where the criterion was 
importance to ASW mission without attack. PREFMAP 
constructed the initial representations to determine which 
of the four models produced ‘'best' representation of the 
TACCO data. The coefficient of variation (17) and the 
Statistical test (F) (Table XIII) were used to determine 
which model is the best representation. 


Table XIII. Significance of Result for Mission 
Without Attack 


MODEL 6 F STATISTIC REJECTION LEVEL 
GENERALIZED ~997 Sore Sau 
WEIGHT DIST. ~949 21.3 <.Ol 
UNWEIGHTED -881 16.7 <ul 
DISTANCE 
VECTOR ~ 795 13.0 <.0l 


| =e oe oe ee ee ee es ee es es es es es ee ae es es es es ee ee es ee ae ee ee ae ee ee ee eee eee eS See eee eee eee 


As with the previous criterion, the null hypothesis 
can be rejected for all four models, Pnaicating that a 
Significant representation of the TACCO ranks were provided 
by all models. 

As before, the four modelS were compared against 
each other to determine the model of best representation. 
The ‘between model' F statistic was used, testing the null 


hypothesis, H that the increase in the proportion of 


Oo! 
variance accounted for by the more powerful model was a 


chance result accounted for by its increased number of 
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parameters. Table XIV show the ‘'between-model' F statistic 


and the associated significance level. 


Table XIV. Comparison of Coefficients of Variation 
for Mission Without Attack 


MODEL — ee ee eee een 
F STAT 
SIGN. 
WEIGHTED 18.7 
-008 
UNWEIGHTED 27.3 4.7 
~003 209 1 
VECTOR 39.6 7.0 6.5 
-002 O1n6 aon 
GENERALIZED WEIGHTED UNWEIGHTED 


There were more pronounced differences in the 
Significance levels between models in Table XIV than in 
Table XI. More specifically, the Generalized Distance Model 
was found to be significantly better in representing the 
TACCO data with a confidence of greater than 99 percent. As 
a result, the Generalized Distance Model was selected as the 
most appropriate model for the no attack criterion. 

As in the previous analysis, the PREFMAP program was 
used to ‘'map' the precise mathematical priority function. 
Using this algorithm, the importance of decision Jj on a 


mission without attack (P (d=) ) was given by: 


woa 
2 a 2 Z 2 _ Z 
+.623x5)%49 +.385x5) X53 0 992K 50X53 


van 








where Xay is the coordinate of decision j on dimension 1 


X59 is the coordinate of decision j on dimension 2 and 
X53 is the coordinate of decision j on dimension 3. 


The values of the Rais aS before, were given in 
Table IX. The priority values for the fourteen decision 


functions are given in Table Xv. 


Table XV. Decision Function Prioritization for Mission 
Without Attack 


PRIORITY NAME PRIORITY 
RANK SCORE 

1 CLASSIFY SIGNAL (CS) ~.2309 
2 DETERMINE SIGNAL IS VALID 

CONTACT (VC) -.2210 
3 DETERMINE TARGET FIX (TF) -.1612 
4 ANTICIPATE TARGET MOVEMENT ~.1473 
5 COORDINATE HAND-OFF (CH) ~.1246 
6 EXTEND SENSOR PATTERN (EP) ~.0637 
7 CREATE SENSOR PATTERN (CP) -.0539 
8 CONSTRUCT SENSOR MONITORING 

PATTERN (MP) -.0523 
9 COMPENSATE FOR PROPAGATION 

CONDITIONS (PC) -~.0456 
10 MANAGE EQUIPMENT AND STORES (ME) 0203 
ip ADJUST PATTERN TO SENSOR 

FAILURE (AP) 0539 
12 GAIN ATTACK CRITERION (AC) .2025 
13 DETERMINE AIRCRAFT WEAPON 

LAUNCH POSITION (LP) .3563 
14 DETERMINE WEAPON AND SETTING 

FOR ATTACK (DS) 4245 


These values represent the prioritization of the 
decirsion functions in the ASW mission without attack on the 
Submarine. For reasons discussed in the previous Section, 


the smaller values represent higher priority in Table XV. 
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V. DISCUSSION 


A. RESULTS 


es summary 


In this thesis. a methodology was developed that 
Perorltlzed ASW decision functions according to different 
Criteria. However, to more fully understand the results of 
the methodology and itS components, the purpose of this 
thesis should be reviewed. The objective of this effort was 
to examine ASW decision making in the S-3A through the use 
of mathematical modeling techniques and to establish a 
prioritization technique for the development of decision 
aids to assist decision making in the system. An examination 
of the S-3A ASW decision making included discussions of the 
history of ASW, the four and six partition spacing of the 
Air ASW mission, and finally the fourteen decision function 
partition of the Air ASW decision space. S-3A TACCOS sorted 
and ranked the 14 decision functions through the use of the 
mathematical modeling techniques of Multidimensional Scaling 
and Unfolding Analysis, a priority mapping of the fourteen 
decision functions was produced. 

The Multidimensional Scaling representation of the 
decision functions was determined to have three orthogonal 


dimensions or axes through a goodness of fit test. The 
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dimensions were interpreted as follows: 
Dimension One - Workload 
Dimension Two — Time Criticality 
Dimension Three - Complexity 

It should be noted that this interpretation of 
dimensional meaning is not unique. The interpretations, toa 
great degree, are subjective in nature relying upon the 
analyst's understanding of the interrelationships of the 
Stimulus set. However, the relative positions of the stimuli 
on each dimension (Figures 8 through 10) iS unique so the 
interpretations of the dimensions should be roughly 
equivalent from analyst to analyst. 

The Unfolding Analysis determined that the 
importance rankings were best represented by the Generalized 
Distance Model in terms of priority functions. Only two of 
these four rankings were utilized in the unfolding analysis. 
The criterion of importance was assumed to be the most 
important feature of the priority mapping, so the criteria 
of importance to the ASW mission with attack and importance 
to the ASW misSion without attack were selected and the 
criteria of urgency and workload rankings were used to 
validate the resulting Priority Mapping algorithm. 

Finally, the results of the Multidimensional Scaling 
program (MINASSA) and the Unfolding Analysis program 
(PREFMAP) were used to ‘map' the precise priority function. 


From the two priority functions, a prioritization of the ASW 
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decision space was established according to both criteria 
(see Tables XIII and XV). 

As a result, a priortization technique for Air ASW 
decision functions was established for the development of 
decision aids in the S-3A. The fundamental question raised 
by the priority ranking shown in Tables XIII and XV is what 


is the meaning of this prioritization of decision functions. 


Zo ENneerpretation of Results 
To better interpret the results of the priority 


mapping function, a relative plot of the 14 decision 
functions is given in Figure 12 for the attack and no-attack 
criterion. The relative position of the decision functions 
on this priority scale for each criterion yields an implicit 
ordering of areas in which TACCOs think decision aids would 
be useful. 
a. Gain Attack Criteria 

The Gain Attack Criteria decision function was 
ranked first in the priority scale for the ‘attack' 
Criterion. Intuitively, this result is not surprising in a 
mission where the goal is attacking the submarine. The 
TACCOs interviewed felt that in a mission where attack on a 
hostile threat was the goal, if attack criteria was not 
gained, the mission would be a failure. 

The implication, from the priority scale of the 
importance to the ASW mission with attack, is that more 


energy should be channeled into the area of decision aids 
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Figure 12. Priority Score of Decision 
Functions (S-3A) 
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concerned with assisting the TACCO in determining when 
attack criteria has been reached. Additionally, Gain Attack 
Criteria ranked highest in the average rankings for 
importance to ASW mission with attack (Table V) and Workload 
(Waele “VIIit) and second highest in the ranking for the 
Uregency (Table VII). This placement distinguishes this 
decision function as time-critical for the TACCO. 

Placement of the Gain Attack Criteria decision 
ereclon in the MDS solution space (Figures 5 througn 10) 
aeoo Commrmms this distifiction. Although this decision 
function is positioned near the middle of dimension three 
(Complexity), it is located at the positive extreme of both 
dimension one (Workload) and two (Time Criticality). This 
Pesttioming inffd@icates a highty time critical and “work 
intensive decision function. Based upon this analysis, the 
decision function - Gain Attack Criteria - represents an 
area in the Air ASW mission where implementation of a 
decision aid would be of great benefit. 


b. Classify Signal and Determine Signal is Valid 


Contact 
The decision functions - Classify Signal and 
Determine Signal is Valid Contact - were ranked first and 


PeeonGmeime the PrIOrity scade Hee Ene ‘no attack’ criterion 
@eeaure 12) and ranked second and third on the priority 
Serpe stom tne ‘attack’ criterion (behind Gain Attack 
Sameer liajy., Memes position on the High extreme of both 


priority scales was unexpected however it was clear that the 
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TACCOs felt these two decision functions to be the heart of 
a successful ASW mission. The TACCOs felt that the entire 
misslon was dependent on the proper evaluation of an 
incoming signal (that could be a hostile threat) and 
Suggested that because this evaluation is So important, 
there is little room for error regarding either decision 
Function. 

The placement of these two decision functions 
(Classify Signal and Determine Signal is Valid Contact) on 
the high extreme of both priority scales suggest that these 
two areas in the Air ASW mission are most important in the 
accomplishment of the mission. The implication is that 
decision aids should be developed for both decision 
functions. Furthermore, both decision functions were ranked 
high in the average rankings for importance to mission with 
attack and without attack. In the average ranking for 
Urgency, both decision functions were positioned in the 
center of the ranking, indicating that Classify Signal and 
Determine Signal is Valid Contact are not critical in terms 
of Urgency. The placement of the decision functions at the 
bottom of the average ranking for Workload indicates that 
both decision functions do not add to the TACCO Workload. In 
the S-3A, Classify Signal and Determine Signal is Valid 
Contact are responsibilities of the TACCO but these decision 
functions are normally delegated to the sensor operators 


with inputs to the TACCO. 
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The two decision functions - Classify Signal and 
Determine Signal is Valid contact - in the sample of the 
fourteen lay at opposite extremes in two dimensions and in 
the center of the third of the MDS space (Figures 5 through 
10). In dimension one, Workload, Classify Signal and 
Determine Signal is Valid Contact lies at the very negative 
extreme representing light workload (Figure 8). 

On the second dimension, Time Criticality, the 
two decision functions are clustered at the ‘Zero' point, 
well away from the other two well-defined groupings 
(Figure 9). This result suggest that Classify Signal and 
Determine Signal is Valid Contact are not as time critical 
as the ‘attack’ functions because the confidence in the 
Contact Validity normally improves over time. However, both 
decision functions are more time critical than the 'tactics' 
functions because confidence improves only after contact is 
established. 

The third dimension, complexity, located both 
decision functions in the positive region indicating a high 
degree of complexity (Figure 10). Many inputs into the 
classification or verification decisions from all sensors 
Suggest that the degree of complexity increases with the 
number of sensors employed. Classify Signal and Determine 
Signal is Valid is a sine quad non. 

The remaining eleven decision functions were not 


addressed further because the apparent order of the three 
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discussed decision functions is not implicitly present in 
the ordering of the last eleven functions. This problem will 
be discussed further in the Conclusion and Recommendations 


sections. 


3. Comparision of Results to Similiar Work 

This analysis has established priority scales for 
both attack and no attack decision situations. The fourteen 
decision functions were ordered in importance by the use of 
Multidimensional Scaling combined with Unfolding Analysis. 

Zachary (1980 a and b) used the same prioritization 
methodology used in this analysis in conjunction with P-3C 
TACCO inputs. the results of his analysis was very similar 
to this analytical result. Although the P-3C is a shore 
based ASW aircraft with a crew of ll, the function of the 
TACCO is very much like that of the S-3A TACCO. 
Consequently, similiar results in the prioritization 
analysis would not be surprising. 

Zachary's prioritization for Importance to the ASW 
Mission with Attack on the Submarine was very nearly the 
same as the S-3A prioritization. These were minor inversions 
of one or two decision functions in the final priority scale 
compariSions, however the differences in the aircraft system 
Capabilities suggest that this result is reasonable. The 
comparision of decision function prioritization for mission 
without attack on submarine provided a similiar result. The 


Priority scales were very similiar except for the inversion 


86 





of one or two decision functions due to aircraft and 
tactical dissimiliarities. Figure 13 shows Zachary's 
priority scale for both attack and no-attack for the P-3C. 
These priority scales are presented in the same format as 
Figure 12 for ease of comparison. 

Zachary's analysis also used three dimensions as 
representative of the MDS decision space (discussed in 
Chapter Four). Zachary's interpretation of his three 
dimensions for the P-3C data was similiar to this analysis, 
however, the placement of each decision function in the 
decision space was markedly different. His dimensional plots 
can be found in Appendix E. This difference could be 
contributed to sampling differences, aircraft differences, 
or training differences between S-3A and P-3C TACCOs. 

Zachary's results reinforces the applicability of 
the results of this analysis. Although two analyses of 
Similiar data do not necessarily validate a new methodology, 
the degree of agreement between results suggest’ further 


application could be of great benefit in the area of ASW. 
B. CONCLUSION 


This thesis has established that a workable mathematical 


technigue exist for the prioritization of Air ASW decision 
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functions, specifically and a general application could be 
established for any complex decision space. The 
prioritization methodology developed in this thesis 
representS a new approach to the problem of assigning 
priorities to decision stimuli. This methodology is based on 
two assumptions: 

1) the decision space is a multidimensional domain 

where all underlying dimensions have some 

potential relevancy to the final prioritization. 

2) inputs to the decision space must be based on 

experience, judgement and intuition. 

The prioritization methodology, Priority Mapping, 
produced two siginificant results. First, the MDS portion of 
the methodology identified the three dimensions which 
underlie the TACCO decisions. Secondly, the ‘'marriage' of 
the MDS results and the Unfolding Analysis provided a 
numerical prioritization of the Air ASW decision space. 

The Priority Mapping algorithm has several interesting 
Characteristics. First, the sorting and ranking positions of 
the data collection is only required once. Repeated testing 
or interviewing of the same subjects is not necessary as in 
many consensus techniques such as Delphi. Ease of data 
collection is a definite advantage of this technique. 
Second, specific as well as general questions can be 
answered by reweighting the parameters of the MDS portion of 


the algorithm (as discussed in Chapter IV). Third, an 
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advantage of this algorithm is the built-in cross checks and 
validations carried out by the interaction of the average 
rankings, MDS unidemnsional plots, and the priority scales. 
In other words, placement of a particular stimulus on the 
priority scale should be reflected in that stimulus’ 
placement on the MDS plots and in the average rankings of 
the test criterion. Additionally, the priority scale and 
average ranking place would have a positive influence on the 
analyst's interpretation of the MDS dimensions. Fourth, the 
Priority Mapping algorithm provides more information about 
the stimulus set than a ranking or an MDS dimensional 
analysis alone. To achieve similar results uSing only 
ranking task or sorting task, the subjects would have to 
agree completely on the placement of each stimuli. The 
algorithm eliminates the need for concensus by combining the 
two techniques into a predictor model. 

The operational application of the priority mapping 
algorithm to fleet assets is where the developmental 
"'pay-off' is received. This technique has advantages that 
unit commanders can appreciate: 

1) ease of data collection - simple forms with 
simple instructions. Any sailor could administer 
the interview with favorable results. 

2) speed of data collection - fast with little 
confusion. The unconstructed sorting and rankings 


for a large stimulus set would take less time than 
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other similiar techniques (i.e. Pairwise MDS) with 
much smaller stimulus sets. 
3) revisit requirements - one data collection visit 
to operationsl units. The unit commander is not 
compelled to provide subjects for data collection 
or verification time after time. 
4) tangible results - a priority scale of the 
Stimulus set. A priority aoa of the stimulus set 
is tangible results of the data collection, 
providing real answers to operational problems. 
Many of the techniques discussed in Chapter II do not 
exhibit these advantages. Fleet acceptance of problem 
Solving methodologies is essential to the development of new 
software and hardware to meet any present or future threat. 
This prioritization methodology shown in Figure 3 can be 
adjusted to fit any complex decision space provided the 
assumptions of dimensional relevency and judgmental inputs 
are not violated. The methodology is flexible enough to be 
used by any branch of the Armed Forces or any Sector of 
industry. Areas of time critical, work intensive, 
multi-faceted decision situations are ideal targets for the 
implementation of the prioritization methodology. The 
implementation of this technique will identify decision 


Situations where decision aids would be most beneficial. 
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C. RECOMMENDATIONS 

The decision functions - Classify Signal and Determine 
Signal is Valid Contact - are at the top of the priority 
Scale of the 14 decision functions defining the Air ASW 
decision space for the S-3A. Both decision functions were 
the highest priority in the no-attack mission and second and 
third highest priority in the attack mission. Therfore, it 
1s recommended that a decision aid be developed to assist 
the S-3A TACCO in the execution of the decision funcitons - 
Classify Signal and Determine Signal is Valid Contact. 

Although the ASW decision space has been prioritized for 
the S-3A, the explicit priority ordering of Classify Signal 
and Determine Signal is Valid Contact across both priority 
scales is not readily apparent in the remaining decision 
function. In other words, it is difficult to assess which of 
the remaining 12 members of the S=-3A ASW decision space 
would be next in priority across both priority scales. 
However, an algorithm for combining the relative weights of 
each priority scale across all decision functions such as 
the Mission Operability Assessment Technique [Helm and 
Donnell, 1978] would solve the problem by providing one 
priority scale across all criterion. 

Therefore, it is recommended that an algorithm be 
developed that would combine priority scalings over all 


relevant criterion to achieve a combinational prioritization 
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of the decision space that would be more useful in the 


development of decision aids. 
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APPENDIX A 


TEXT OF INTRODUCTORY BRIEFING GIVEN TO S-3A TAGCOS PRIOR 
TO MULTIDIMENSIONAL SCALING INTERVIEWS 

I would like to begin by thanking you all for coming 
here today to help us in this effort. You've been asked to 
come here because of your familiarity with ASW and ASW 
decision making. Before we ask you some specific questions 
about ASW, I'd like to begin by providing a brief background 
on what it is that we mean by decision aids and decision 
aiding, what the overall structure of our program is, where 
we have been, and why we have come here to talk to you. AS 
good a place as any to begin is with the definition of a 
decision aid. 

Very simply put, a decision aid is any kind of device 
that helps humans make better, more efficient, clearer, and 
faster decisions. Now, obviously, a wide range of possible 
things can be considered decision aids - from a pencil and a 
Sheet of paper which enable you to do calculations to large 
computer systems and programs. What we're most interested in 
are specific tools that will enable you as a TACCO to 
interact with your on-board computer system to help you the 
kinds of decisions that you have to make in the course of an 
ASW mission. 


The Navy's interest in decision aiding has been 
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increasing significantly recently because of the realization 
that warfare, in general, and ASW, in particular, is 
becoming more automated and more highly technological. The 
Speed and complexity with which decisions must be made is 
increaSing constantly to a point where, in the not too 
distant future, you as TACCOs, will be overloaded, possibly 
beyond your capability to make necessary decisions within a 
reasonable amount of time. Therefore, the Navy is interested 
in developing computerized systems of decision aids that 
will help you keep pace with the increasing automation 
on-board your aircraft. It should be pointed out that these 
decision aids will not take you out of the decision making 
process or automate your functions. Rather, they will 
provide you with better, more intelligent Support from 
machines and will give you time to do what you do best -- 
think and make decision. The whole concept of decision 
aiding is based on the notion that the most complex, useful, 
and important piece of equipment on any platform is the 
human-brain. Humans are on-board to make decisions; but 
computers and other kinds of devices can assist by managing 
information, making certain kinds of data available at your 
fingertips, helping you remember things (like the procedures 
you must go through to accomplish a specific function), 
performing certain kinds of calculations for you, and So 
EOREN . 


Decision aids can work at a variety of levels. Simple 
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decision aids can, for example, provide very rudimentary 
bookkeeping functions or provide checklists of things that 
must be done. More complex decision aids can "ask" you the 
questions that you should be asking yourself. Still more 
complex decision aids can anticipate some of the information 
that you may want, based on your past performance; they can 
perform certain kinds of calculations automatically so that 
the results of the calculations will be available when 
requested; they can make certain kinds of inferences about 
what might happen in the future enabling you to play "what 
if" games to find out what results might be obtained if a 
Specific course of action is undertaken. These latter 
things, which are at the higher end of the spectrum of the 
capabilities of decision aids, are the ones that we're most 
interested in. 

To give you a feeling for the types of aids that are 
possible, I'd like to review some of the other decision aid 
Projects of this type that are on going today in the Navy. 
The largest decision aiding effort to date has been 
undertaken by the Office of Naval Research (ONR). Their 
Program has concentrated on developing a variety of decision 
aids for carrier-based air strike operations. Some of the 
specific problems to be addressed by these decision aids 
are; planning the ingress route for an incoming 
carrier-based air strike or reconnaissance mission through a 


complex sensor field; determining the specific timing of 
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alpha type air strikes; task force EMCOM planning; and 
overall air strike campaign planning. The decision aids 
\ 

being developed by ONR are very high level decision aids, 
both in the sense that they provide a great deal of 
assistance to the human and in the sense that they deal with 
very high level command and control decisions (at the task 
force commander level or higher). 

More germaine to our discussion today is the effort that 
we have undertaken under the joint sponsorship of the Naval 
Air Development Center and the Office of Naval Research to 
identify possible decision aids for on-platform ASW 
operations. I'd like to review some of the earlier parts of 
this effort to clarify why we are here today. 

We began our effort by looking at the ASW platforms that 
will be in use in the 1980-1985 timeframe - the P-3C, the 
S-3A, and the LAMPS MK-III. We examined the specific 
missions that are undertaken by these three platforms in 
order to define a generic or generalized ASW mission, to 
identify commonalities in the missions flown by the three 
platforms, and to identify some of the critical difference 
both in crew functions and in the details of the missions 
that were undertaken. We then constructed a flowchart of the 
Sequence of operations that takes place in the generalized 
mission. The mission was subdivided into very broad 
categories — movement to the search area, on-station search, 


Prosecution of the contract, possibly culminating in attack 
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and destruction of a hostile submarine. These states were 
then subdivided into the detailed steps that involved 
specific decision about classification and sensor extension 
of search area, and anticipation of target movement. We 
identified the specific sequences in which these decisions 
were made, recognizing that the interrelations between these 
decisions are highly dependent on the sequential nature of 
the ASW mission. The decisions that take place in the attack 
phase, for example, are dependent upon the succesful 
completion of those portions of the mission that relate to 
search and early prosecution of a contact. Ultimately, we 
identified six broad areas that we termed decision making 
Situations. We defined a decision making situation as a 
portion of a mission in which complex sequence of decisions 
has to be made by the TACCO. These situations were complex 
because they involved trading off a number of factors 
against one another. The six decision situations were: 

1) On-Station Search, 

2) Contact Classification/Verification 

3) Target Localization, 

4) Surveillance Tracking, 

5) Lost Contact Reacquisition, and 

6) Attack Planning. 

The next issue we addressed in our program was the 
relative priority of each of these situations. By 
prioritizing the decision situations, we felt that we were 
immediately confronted by the problem that there was no 


Single dimension, or criteria, by which we could prioritize 


the decision situations. They obviously are influenced by 
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their sequence in the mission. They are also influenced by 
varying time constraints on the decisions that must be made 
in each situation and how busy the TACCO is in each 
Situation. The more we thought about it, the more different 
Criteria for prioritization we were able to define. It 
became clear to us that one of the problems was that we, as 
analysts, could not determine the prioritization. We decided 
that the only way to determine the specific criteria that 
were relevant to the prioritization of these decisions was 
to ask the people who made these kinds of decisions, people 
such as yourselves. That is why we are here today. We want 
to determine how the various decisions that you, as TACCOs 
make, should be pritoritized. To do this, we have to 
determine the dimensions or criteria by which these 
decisions are interrelated in your minds. Then we have to 
determine the importance or priority of these decisions in a 
mission. 

There are a number of techniques that can be used to do 
this. One way would be to ask you, in very lengthy and 
detailed discussion, to try to identify the dimensions which 
are salient to ASW decision making for you. But besides 
taking a lot of your time, it is not clear that the 
technique would work. People are often very unclear about 
the underlying principles they use to think about common, 
everyday things, like decision making. In addition, we 


would have the problem of resolving the differences we 
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encountered between the various people we talked to. So 
instead, we have decided to use a more formal mathematical 
technique known as multidimensional scaling which will take 
less of your time and will enable us to determine both the 
dimensions and the relative importance of the various 
decisions from the same set of data. Multidimensional 
scaling uses the computer program to calculate the 
dimensions from very simple judgements made by you about the 
basic similarity or difference among these various 
decisions. We also decided that we wanted to address not 
Just the broad analytic catefories that we have called 
decision situations, but more precise, meaningful, specific 
decisions that are made by TACCOs. We identified 14 of these 
decisions, many of which occur in several of the decision 
Situations. There are, of course, many more decisions that 
are made in the course if a miSsion but the 14 decisions 
that we chose were ones that appeared in more than a single 
decision situation or ones that seemed particulary amenable 
to decision aiding. 

You have in front of you a set of cards.- Each card 
describes one of these decisions. We're going to ask you to 
make judgements about which decisions you feel are similar 
or dissimilar, and to rank them by various criteria. The 


results of these judged similiarities and rankings will be 


Card-packs describing the 14 decision functions passed out 
to interviewees at this point. 
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used by the multidimensional Scalinguempuocess —~=tO 
Mathematically determine a set of relationships between 
these decisions that will help us in prioritizing them. It 
will also help us to understand the kinds of distinctions 
that you find most relevent among these various decisions. 
But, most importantly, we feel it will enable us to relate 
our analysis in a concrete way to your knowledge, 
experience, and intuition of the ASW missions. With your 
help, we will be able to determine the best places to apply 
decision aiding techniques in the ASW mission. 

I'd like to add one other note about the way in which 
you should sort and rank these decision functions in later 
portions of this procedure. Do not sore OG. Gankoeeae 
decisions in terms of how you think a decision aid could 
help you make these decision nor in terms of how you think 
they could be made better, but rather in terms of how you 
currently go about making these decisions and how they 
Currently are handled on the platforms on which you have 


worked. 
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APPENDIX B 


RESPONSE FORMS FOR INTERVIEWS 


This appendix contains the forms on which the TACCO's 
interviewed at NAS Cecil Field recorded the results of the 
unconstrained sorting and four rankings. All of the pages in 
the appendix were given to each interviewee as a stapled 
packet. The first page was used to record some general 
biographical information on the respondent. The next two 
pages provided the instructions and response sheet for the 
unconstrained sorting of the ASW decisions. The remaining 
pages provided the instructions and response forms for four 


different rankings of the fourteen decisions. 
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BIOGRAPHICAL INFORMATION 





Aircraft Type: 


Rank/Designator: 





Organization: 


Date TACCO Designation: 





Date Mission Commander (if applicable): 


Deployment Locations as TACCO (and dates): 


eT 
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INSTRUCTIONS FOR THE 'SORTING" TASK 


You have been given a pack of fourteen yellow caards. On 
each of these cards is a decision or problem that is 
encountered in an ASW mission. You have all probably faced 
these problems many times in your experience as TACCOs. Each 
decision is in some way different from all the others, but 
each decision is not totally unique; some of the decisions 
are more alike than others. What we would like you to do is 
arrange these decisions and problems into groups according 
to how similar they are. That is, if these are a number of 
cards which represent decisions or problems you feel that, 
based on your experience, are similar, then place all these 
cards together. If there is a card which you think is 
sufficiently unique that it isn't similar to any of the 
others, then place it in a group by itself. There is no 
limit on the number of groups you can make or on the number 
of cards you can place in each group. While the final 
definition of what constitutes similar decisions is left to 
you, we would like you to think of it as referring to 
decisions or problems which somehow solve in the same way or 


which place similar demands on you as TACCO. 


104 








INSTRUCTIONS: Record each group of decisions you have formed 
on a Separate block below. Take one group and write the code 
for the decisions in it on the blank line provided. Then do 
the same for another group in the next block until each 
group has been recorded in a separate block. If you wish, 
you may also include a short phrase describing the 


Similarity you saw in that group of decisions. 








> ae Ge ee ce ae Ge em em cD cD aD cD cD cD > cm aD cee GED cD cD cD cD aD cD aD cD cD cD cD cD cD cD cD cee GD GE cD CD Gm ae ce cD cm ce ce a= Ge cee GE ce cm cme ae cm ae eee oe a= 
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INSTRUCTIONS FOR RANKING 1 


We would like you to rank these decision problems in the 
order in which a less-than-optimal decision would have the 
most detrimental effect on the mission. The decision problem 
for which a less-than-optimal decision would have the least 
detrimental impact on the mission should be ranked last, and 
the decision problem for which a less-than-optimal decision 
would have the greatest detrimental impact on the mission 
Should be ranked first. For the mission, assume that it 1s 
an ASW mission in which the submarine is to be attacked and 


destroyed if possible. 
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INSTRUCTIONS: Enter the two-letter codes for the decisions 
below in the order in which you ranked them. That is, write, 
the code for the decision you ranked first next to '1l.', for 
the decision you ranked second next to '2.' and so on. 


l. 
2s 


NO. 
12. 


13. 
a. 
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INSTRUCTIONS FOR RANKING 2 


Rank the decisions according to the same criterion as in 
the previous case, but for the mission, assume as ASW 
mission in which the submarine is only to be tracked and 


handed-off to a relief platform. 
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INSTRUCTIONS: Enter the two-letter codes for the decisions 
below in the order in which you ranked them. That is, write 
the code for the decision you ranked first next to ‘'l.', 


for the decision you ranked second next to '2.' and So on. 
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INSTRUCTIONS FOR RANKING 3 


Rank the decisions according to their urgency. Define 
urgency as referring to the speed with which the decision 


has to be made once you know it must be made. 
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INSTRUCTIONS: Enter the two-letter codes fir the decisions 
below in the order in which you ranked them. That is, write 
the code for the decision you ranked first next to 'l.‘, for 


the decision you ranked second next to '2.' and So on. 


10. 


WN 


12. 
13. 
14. 


ea 





INSTRUCTIONS FOR RANKING 4 


Rank these decisions according to your workload during 
each of them. Consider both your cognitive, or mental 
workload and your physical workload in ranking’ the 
deicisons. Rank the decision during which your workload is 
heaviest first, and the one during which your workload is 


least heavy last. 
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INSTRUCTIONS: Enter the two-letter codes fir the decisions 
below in the order in which you ranked them. That is, write 
the code for the decision you ranked first next to 'l.', for 


the decision you ranked second next to '2.' and So on. 


1. 
2 
B0 
a 
le 
6. 
We 
8. 
9. 

10. 
ll. 
2° 
13. 
14. 


a 
SE 
i 


eS 





APPENDIX C 


RESULTS OF UNCONSTRAINED SORTINGS OF ASW DECISIONS 


This appendix presents the results of unconstrained 
Sortings of fourteen air ASW decision by 30 S-3A Tactical 
Coordinators (TACCO'sS) stationed at NAS Cecil Field, 
Florida. The sortings were performed as part of interviews 
conducted between 10 December 1979 and 13 December 1979. The 
interview procedure is described in Chapter 3. 

The fourteen decisions that were sorted are shown in 
Table XVI. The instructions for the sorting task, and a 
Sample of the form on which the resSultS were recorded, are 
given in Appendix B. 

Each sorting presented as several lists of decisions. In 
this task, each decision function is identified only by a 
two-letter code. The code used for each decision is also 


given in Table XVI. 
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Table XVI. Fourteen ASW Decision Function 
and Alphabetic Codes 


CODE DECISION FUNCTION 

AP ADJUST PATTERN TO SENSOR FAILURE 

EP EXTEND SENSOR PATTERN 

AM ANTICIPATE TARGET MOVEMENT 

MP CONSTRUCT SENSOR MONITORING PATTERN 

CH COORDINATE HAND-OFF 

vc DETERMINE SIGNAL IS VALID CONTACT 

DS DETERMINE WEAPON AND SETTING FOR ATTACK 

AC GAIN ATTACK CRITERIA 

Re: DETERMINE TARGET FIX 

CP CREATE SENSOR PATTERN 

ME MANAGE EQUIPMENT/STORES TO ACCOMODATE 
PRESENT AND FUTURE NEEDS 

CS CLASSIFY SIGNAL 

PC COMPENSATE FOR IN SITU ACOUSTICAL 
AND ATMOSPHERIC PROPAGATION CONDITIONS 

LP DETERMINE AIRCRAFT LAUNCH POSITION FOR 


ATTACK ON TARGET 


SUBJECT 1 
GROUP DECISION FUNCTIONS IN GROUP 
ME 
CP,MP,PC,AP,EP 
vc,cs 
TF,AM,AC 
DS, LP 
CH 


NM & W NE 


SUBJECT 2 
GROUP DECISION FUNCTIONS IN GROUP 
CP,PC,MP,ME 
EP,AP 
vc,cs 
TF,AM 
AC, LP,DS 
CH 


Om & WN Ee 


ba 








SUBJECT 3 
GROUP DECISION FUNCTIONS IN GROUP 


1 CP,ME,AP,MP 
2 EP,PC 
3 vc,cs 
4 TP,AM 
5 AC,DS, LP 
6 CH 
SUBJECT 4 
GROUP DECISION FUNCTIONS IN GROUP 
1 CP,MP,EP 
2 PC,ME,AP 
3 Mc ,CS,TF 
4 AM 
5 Bowl ,Ac 
6 CH 
SUBJECT 5 
GROUP DECISION FUNCTIONS IN GROUP 
1 AM,CP,MP 
2 PC,AP,ME,EP 
3 vc,Ccs, TF 
4 AC,DS, LP 
5 CH 
SUBJECT 6 
GROUP DECISION FUNCTIONS IN GROUP 
1 @p,PceMP,EP  ~=~——~C~SCS«SC 
2 ME,AP 
3 vc ,cS,TF 
4 CH 
5 AM,AC,DS, LP 
SUBJECT 7 
GROUP DECISION FUNCTIONS IN GROUP 
1 AM,CP,MP,EP,DS 
2 PC 
3 CS,TF,AP 
4 ME 
5 vc 
6 AC, LP 
7 CH 
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SUBJECT 8 
GROUP 

i 

2 

3 

“ 

5 


SUBJECT 9 
GROUP 


Wn &W NH 


SUBJECT 10 
GROUP 

1 

2 

3 

_ 

5 


SUBJECT ll 
GROUP 


nM &W NH 


SUBJECT 12 
GROUP 


Wm & WN 


DECISION FUNCTIONS IN GROUP 
CP,ME,PC,MP,EP,AP 
TF,AM,LP,AC 

cs,Vvc 

DS 

CH 


DECISION FUNCTIONS IN GROUP 
PC,MP,CP,AP,EP 

LP,AC,DS 

es,Vvc 

DS 

CH 


DECISION FUNCTIONS IN GROUP 
TF,AM,LP,DS,A 

Be,VvC,Co 

MP,AP,ME 

CP TEP 

CH 


DECISION FUNCTIONS IN GROUP 
CP,MP,EP,PC,AP 

DS,AC, LP 

wencs 

TF,AM 

ME 

CH 


DECISION FUNCTIONS IN GROUP 
CS,VC 

AC,DS, LP 

MP,PC,ME,AP 

TF,AM 

CH 

Cr, Le 


7, 





SUBUECT 13 
GROUP 


wn & WN Fe 


SUBJECT 14 
GROUP 


Nr 


SOM & W 


SUBJECT 15 
GROUP 


in & WD Fe 


SUBJECT 16 
GROUP 


in & WN Fe 


SUBJECT 17 
GROUP 


in & WNr- 


DECISION FUNCTIONS IN GROUP 
Gsmve 

DS, LP,AM,TF,AC 

CH 

AP,EP,MP,PC,CP 

ME 


DECISION FUNCTIONS IN GROUP 
AC,AM,DS,TF 

CP,EP,AP 

MP, LP 

VC,PC 

CS 

ME 

CH 


DECISION FUNCTION IN GROUP 
CP,PC,MP,ME,EP,AP 
LP,AM,TF,AC 

GS..VC 

DS 

CH 


DECISION FUNCTIONS IN GROUP 
PC,CP,AP,MP,EP 

LP,DS,AC,AM 

Conve, Le 

ME 

CH 


DECISION FUNCTIONS IN GROUP 
ME,AP,DS,CH 

PC,CP,AM,TF,AC 

MP,EP 

CS,VC 

CH 
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SUBJECT 18 


GROUP DECISION FUNCTIONS IN GROUP 
J AP,EP,MP,PC,CP 
2 DS, LP,AM, TF,AC 
3 CS,VC 
4 CH 
5 ME 
SUBJECT 19 
GROUP DECISION FUNCTIONS IN GROUP 
1 CP,PC,MP,EP 
2 AM,AC,DS,LP 
3 ME,AP 
4 VC,CS,TF 
5 CH 
SUBJECT 20 
GROUP DECISION FUNCTIONS IN GROUP 
1 CP,EP 
2 MP,PC,ME,AP 
3 AC,DS,LP 
4 TE,AM 
5 cs,VvC 
6 CH 
SUBJECT 21 
GROUP DECISION FUNCTIONS IN GROUP 
Ih CP,MP,EP,PC,AP 
2 DS,AC,LP 
3 TF ,AM 
a ME 
5 vc ,CS 
6 CH 
SUBJECT 22 
GROUP DECISION FUNCTIONS IN GROUP 
1 PC,ME,AP 
2 CP,MP,EP 
3 VG,Co, TF 
4 DS,LP,AC 
5 AM 
6 CH 
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SUBJECT 23 
GROUP 


un & WN FH 


SUBJECT 24 
GROUP 


—~ 


OU & WN 


SUBJECT 25 
GROUP 


Om & Ww Nr 


SUBJECT 26 
GROUP 


mn & W NF 


SUBJECT 27 
GROUP 


wn & WN re 


DECISION FUNCTIONS IN GROUP 
AC,DS,LP,AM,TF 

MP,AP,ME 

PC,VC,CS 

CP,EP 

CH 


DECISION FUNCTIONS IN GROUP 
AC, DS,LP 

CP,ME,AP,MP 

EP,PC 

TF,AM 

Viens 

CH 


DECISION FUNCTIONS IN GROUP 
PC,MP,CP,EP,AP 

LP,AC,DS 

ME 

VC,CS 

AM, TF 

CH 


DECISION FUNCTIONS IN GROUP 
ME,MP,PC,CP 

AC, LP,DS 

EP,AP 

Wic7ZcS 

TF,AM 

CH 


DECISION FUNCTIONS IN GROUP 
AP,EP,MP,PC,ME,CP 

AC, LP,AM, TF 

VG,CS 

DS 

CH 
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SUBJECT 28 
GROUP DECISION FUNCTIONS IN GROUP 
CP,MP,PC,EP,AP 
TE,AM,AC 
ME 
CS,VC 
LP,DS 
CH 


OWN &WNF 


SUBJECT 29 
GROUP DECISION FUNCTIONS IN GROUP 
CP,ME,EP,TF,AM,AP,LP,MP 
vc,cs 
PC 
DS 
AC 
CH 


Ou — WN FE 


SUBJECT 30 
GROUP DECISION FUNCTIONS IN GROUP 
CP,ME,AP,MP 
DS, LP,AC 
vc,cs 
TF,AM 
PC, EP 
CH 


N ke 


OV UI om WwW 
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APPENDIX D 


RESULTS OF RANKINGS OF ASW DECISIONS 


This appendix presents the results of rankings of 
fourteen Air ASW decisions (listed in Table XVI) by 30 
TACCOsS from NAS Cecil Field. These rankings were performed 
aS part of the interview described in Chapter 3. The 
fourteen decisions were ranked according to four different 
criteria. The instructions for these rankings and the forms 
on which the results were recorded are given in Appendix B. 
The first ranking was done according to the perceived 
‘importance’ of the decision in a mission where the 
objective is to ‘attack’ and destroy the hostile submarine. 
The second ranking was done according to the perceived 
‘importance’ of the decision function in a mission where the 
objective is to survey the submarine only. The third rankign 
was done according to the preceived ‘urgency’ of the 
decisions in whatever type of mission gave then the greatest 
urgency. The fourth ranking was done according to the 
TACCO's perceived ‘workload' (both cognitive and physical) 
during each of the fourteen decision functions. 

The results of these four rankings are given below. The 
ranking generates by each individual using each criteria is 


presented as a list of the two-letter codes used in the 


ZZ 








interviews to represent the decision functions. Table XVI 
(Appendix C) contains the full decision name represented by 
each two-letter code. The order in which the individuals are 


listed is arbitrary, but is the same for all four rankings. 
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RANKING 1 
CRITERION: IMPORTANCE TO MISSION WITH ATTACK ON SUBMARINE 


SUBJECT DECISION FUNCTION (RANKING GIVEN) 
NUMBER 


ee ae DE RD DS ce a ee es es es ee es we ee ee ew es oe oe oO a“ oe oe oe 2 oe oe 2 om oe ee ee oe oe oe oe oe oe oe oe oe oe 


it 7 8 Z Dynes Dale 3 Ik 4 14 10 6 12 
2 i. 10 4 11 6 8 2 1 5 14 2 a2 3 
3 6 5 10 3 14 av eo eee 2 1 & 8 2. 22 
4 WZ 13 7 8 14 z 2 1 oF LOe eh! 5 9 3 
5 2 12 8 13 14 2 4S i 7 ill 10 3 6 5 
6 ieee LZ ] 4 14 5 9 8 y, S003 6 Zand 
7 el 6 2 7 aks 8 + i 5 10 14 oie 2 3 
8 12 6 2 3 14 8 4 10 2 NG sas 7 Sa 
9 Sues 1 2 5 11 14 10 4 i 3 7 8 2 6 2 
HOD 10 060OC«d13 6 12 14 Zo kel 5 4 3 7 ut 8 2 
11 a giz 15 Bales 7 3 2 4 11 14 6 10 1 
eZ Ss wal 6 12 13 £10 7 4 3 Zz 2 1 14 5 
a3 2 > WO 14 12 3 - 5 ase di 8 2 
14 13 #10 6 9 14 2 4 8 7 eZ Sm 5 
ss, oye 5 Ss 13 4 7 8 6 2 14 1 2s E20 
fm. ll 12 6 9 14 8 4 2 3 apes Speke i 
uO 9 it 8 10 14 13 141 i 5 3 Bit 4 2 
18 11 a 10 Sete Sree 2 4 3 6 1 14 8 7 2 
1 oF i di 8 14 6 zs 4 Se t2 13 eo 3 
ZO il2 it 7 9 14 1 5 4 3 10 13 2 8 6 
ae i 86 4 10 14 2 8 3 7 Sali 1 6 2 
Ze ke 3 8 10 14 2 6 5 a 9 a 32) len i 
Zee lS 612 8 10 14 - 6 Z 5 iis 3 9 1 
24 14 #10 > 12 6 2 8 4 Se ol 1 7 9 
Zo, 10. 11 8 Soe, 14" is 3 1 7 5 “ Sm iz Z 
26 6 7 12 10 14 #213 3 2 9 5 8 J 4 
Za EC 2 S13 3 14 6 9 8 7. 2 4 5 ee 
238 2 “10 Wet 4 2 9 3 6.13 8 1 14 5 
Zo 12 7 4 14 9 1 6 3 Sail. 2S 542 10 8 
30 8 7 4 ll 5 14 2 3 6 10 Sa e)3, 8-2 1 
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RANKING 2 
CRITERION: IMPORTANCE TO MISSION WITH TRACKING OF SUBMARINE 
BUT NO ATTACK 


SUBJECT DECISION FUNCTION (RANKING GIVEN) 
NUMBER 


1 4 5 9 2 10 @ 14 12 8 Be i Z oe ois 
2 ll 8 2 9 5 6 14 i 3. 12 a » lO) 13 
3 6 a 640 Ss 1 7 14 (12 9 1 4 8 Z 13 
“i oF 10 Z 8 3 4. “13-12 i 7 wel 6 5 14 
5 v.16 oe LL 6 1 12 14 4 9 8 2 a ei3 
6 9 8 1 4 10 5 ea 2 id 3 eet 6 2 14 
i 7 oy at 2 4 > 24 12 9 ie 6 BS. Is 
8 ll 5 2 S 9 1 As, Ia 8 - 10 6 4 12 
9 el 8 2 9 d 6 14 13 i Lo 3 5 4 12 
10 ao if 8 10 5 2° Va 212 = 3 6 i es 
L 7 10 2 6 1 OP da oe ele 3 See 4 8 14 
ez. 12 2 5 10 6 ll 14 4 3 2 a 1 oe - 13 
13 p17 4 10 2a Zo Sale 3 5 9 6 if S13 
ay ll 8 > 7 4 Zea 2 6 eo 3 so] | hes 
for lt 13 6 4 9 S 8 7 5 1 14 2° ( b2e ALO 
HS < 10 £ 2 8 3 oa 2 1 (ey 4 6-343 
la e] 4 5 OleOle lel eles 8 3 1 Poe Bepl3 
18 qi 6 9 eo 4 14 13 10 IL 8 5 Sel 2 
i 12 8 4 6 i} 5: pals 24°20 9 2 heey 
20 10 9 - 7 5 Ty Aree 2 3 Sa 2 Or Ls 
Ze 12 13 >.) Ll 4 2 2 3 8 6 14 i as 0 
22 a lg 5 i Le 2. ee lZ 4 6 i 3 8 14 
23 Ss 1s 3 14 5 4 10 9 2 t) 6 Lee 
24 11 8 4 10 5 2 wees eZ 3 9 7 1 6 14 
25 9 8 6 iy PS 211 Sisal Z 2 S 2 4 10 14 
26 6 7 3 9 4 10 13 12 2 5 8 Te Pile ka 
2a 7 10 9 3 8 6 14 12 121i 2 4 5 i us 
238 ul 6 oF 610 3 2, 13, 12 Wee 8 4 i 9 14 
zg 9 6 Se Lt 5 eles cele 4 8 10 2 7 14 
30 6 5 2 8 3 9 13 12 4 ii 7D LO as 
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RANKING 3 


CRITERION: URGENCY OF THE DECISION FUNCTION WITHIN A MISSION 


SUBJECT DECISION FUNCTION (RANKING GIVEN) 
NUMBER 


1 2 8 4 10 14 iL 6 5 3 oe Ze 7 
2 2 8 a Ee 6 4 Z 3 7 #14 #211 £10 212 a 
Se. 1 2 > gs 14 7 2 5 4 12 10 6 8 it 
og 9 8 4 10 14 i 7 5 3° ils LZ 2. <3 6 
5 te atie—10° 13 8 4 3 1 9 14 12 5 6 Z 
6 = 5 S. ls 9 10 6 - Ze a4 2 elt 8 1 
uy 6 Z 4 11 14 8 2 3 Be) 2 eS 29> 20 i 
8 8 7 4 13 14 9 3 2 Opel 2 au Bel: ee 1 
9 1 Sate 6 64 4 7 6 5 13 8 3 10 2 
10 3 9 or... 13 2a ei 1 7 5 4 10 8 7 
she 23 4 14 10 1 3 8 9 2 See 6 12 
12 ee 2 a 13 5 7 i Soe lbie 14 4 10 6 
ms 12 «ili 2) ee) 9 4A 3 i ea 8 6 10 2 
14 oe IZ 4 11 14 8 5 2 3.13 7 9 Eo 1 
‘> 11 10 a) 13 9 6 2 1 4 12 8 7 14 3 
moe 2 (le 3 9 14 2 aS 4 7 Selo 1 6 5 
iy 12 «a Sls =~ CL To 3 ii 6 7 5 2 4 2 
18 8 9 yao! 12) ok 3 4 2 omer 5 14 6 a 
19 6 2 Deo) Ld 7 “ 2 3 14 10 o= gi2 a 
20 12 10 6 ll 14 4 7 i 5 3- 13 3 9 2 
21 yo 5) Open Ll 2 1 4 8 14 12 6 3 
22 7 #14 > 2. 13 3 8 i 4 11 9 eo =k0 2 
25 13 414 ym WO”. 12 4 3 Z Sy. 1) 8 5 9 Ih 
24 8 13 7 #14 12 4 3 2 6 oe 5 10 1 
25 i «6B 3 14 #11 7 2 5 4 12 8 10 6 Z 
26 14 12 4 13 8 2 9 6 Se 7 eo 5 
27 2 (m2 4 11 8 5 3 iL 7 14 «10 6. -LES 2 
28 10 9 Soe 13 6 4 2 = ee es 8 We oe 1 
29 i2 8 Ames. 1 2 2 3 5 6 Jey 24 1 
30 Omerl = 812 613.610 5 Z 1 9 8 14 a 7 3 
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RANKING 


4 


CRITERION: WORKLOAD OF TACCO DURING DECISION FUNCTION 


DECISION FUNCTION (RANKING GIVEN) 


<a OEP a PP GP ee ee ee ee ee ee ee ee ee ee ee es ee ee ee ee ee ee ee ee ee ee Se ee es ee es es ee es eo 


SUBJECT 
NUMBER 
AP 
i 6 
2 8 
3 10 
> Pe 
5 * 
6 9 
7 7 
8 7 
2 2 
10 2 
ll 14 
i213 
iS 5 
14 10 
iS ~ 
16 6 
17 14 
rs. 10 
i 10 
20 i 
Zee 
2211 
23 sal 
24 4 
25 8 
26 9 
27 Ave 
28 
29 14 
30 8 
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APPENDIX E 


PLOTS ON DECISION FUNCTIONS FROM THREE DIMENSIONS 
(FROM ZACHARY 1980b) 
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